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The  initial  co-sponsored  Air  Force  Systems  Command/Naval 
Material  Command  Science  and  Engineering  Symposium  was  held  at  the 
Naval  Amphibious  Base,  Coronado  on  16  -  19  October  1978.  The  theme 
of  the  1978  Symposium  was  "Advanced  Technologies  -  Key  to  Capabilities 
at  Affordable  Cost." 

The  objectives  of  this  first  joint  Navy/Air  Force  Science  and 
Engineering  Symposium  were  to: 

.  Provide  a  forum  for  military  and  civilian  laboratory 
scientific  and  technical  researchers  to  demonstrate 
the  spectrum  and  nature  of  1978  achievements  by  their 
services  in  the  areas  of 

.  Armament 
.  Avionics 
.  Basic  Research 
.  Flight  Dynamics 

.  Recognize  outstanding  technical  achievement  in  each 
of  these  areas  and  select  the  outstanding  technical 
paper  within  the  Navy  and  the  Air  Force  for  1978 

.  Assist  in  placing  the  future  Air  Research  and 

Development  of  both  services  in  correct  perspective 
and  to  promote  the  exchange  of  ideas  between  the  Navy 
and  Air  Force  Laboratories 

.  Stress  the  need  for  imagination,  vision  and  overall 
excellence  within  the  technology  community,  assuring 
that  the  air  systems  of  the  future  will  not  only  be 
effective  but  affordable. 

Based  upon  the  success  of  the  initial  joint  symposium  (which 
was  heretofore  an  Air  Force  event),  future  symposia  are  planned  with 
joint  Navy/ Air  Force  participation. 


.  Human  Resources 
.  Materials 
.  Propulsion 


SR 
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HUMAN  RESOURCES  RESEARCH  AND  DEVELOPMENT 


BY 


Herbert  J.  Clark,  PhD 


Plans  and  Programs  Office 

Air  Force  Human  Resources  Laboratory 
Brooks  Air  Force  Base ,  Texas 


Introduction 


During  tomorrow's  session  on  Human  Resources,  you  will 
hear  six  Air  Force  papers;  four  from  the  Air  Force  Human 
Resources  Laboratory  (AFHRL)  and  two  from  the  Aerospace 
Medical  Division  (AMD) .  Both  organizations  are  more  people 
oriented  than  hardware  oriented,  and  both  contribute  to  Air 
Force  readiness  through  the  development  of  new  technologies 
and  devices  designed  to  increase  man's  productivity  and  safety 
My  purpose  today  is  to  provide  you  with  a  broad  overview  of 
the  R§D  programs  of  each  of  these  organizations. 


AF  Human  Resources  Laborator> 


in<:o  Human  Resources  Laboratory  (HRL)  was  established  in 
1968  and,  since  that  time,  has  had  an  R§D  mission  in  the  areas 
pf  pponnel  utilization,  flying  training  technology,  and 
technical  training  technology.  For  the  past  few  years,  the 
emphasis  has  been  on  flying  training  technology,  especially 
the  design  and  utilization  of  advanced  flight  simulators. 


I?®  six  divisions;  three  are  colocated 

With  the  HQ  at  Brooks  AFB,  Texas.  These  three  are  the  Per¬ 
sonnel  Research  Division,  the  Occupation  and  Manpower  Research 
Division,  and  the  Computational  Sciences  Division.  All  con¬ 
duct  RgD  primarily  in  the  area  of  personnel  utilization. 


Flying  training  and  technical  training  R8D  is  conducted 
principally  at  the  Flying  Training  and  Technical  Training 
Williams  AFB,  Arizona  and  Lowry  AFB,  Colorado. 
The  Advanced  Systems  Division  at  Wright-Patterson  AFB,  Ohio 
also  contributes  new  technologies  to  the  areas  of  flying 
training  and  technical  training  through  the  development  of 
advanced  simulator  designs,  techniques  for  improving  the 
performance  of  maintenance  technicians ,  and  methods  for  pre- 
dicting  the  human  resources  requirements  of  new  weapon 
systems . 


All  AFHRL  R§D  is  supported  with  6.1,  6.2,  and  6.3  funds. 
The  personnel  authorization  is  367  positions  requiring  the 
talents  of  psychologists,  engineers,  computer  scientists, 
econometricians,  and  educational  technicians.  Two  hundred 
and  thirty-three  employees  have  degrees,  56  of  them  PhDs. 

,  Personnel  Utilization.  The  primary  R§D  objective  in 
area  of  personnel  utilization  is  to  optimally  match  the 
person  to  the  job.  This  task  is  particularly  challenging 
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at  the  present  time  because  the  Air  Force  is  faced  with  a 
declining  manpower  pool,  high  rates  of  attrition,  and  an 
increased  need  for  superior  job  performance  to  support 
increasingly  complex  weapon  systems.  Since  Air  Force  jobs 
require  a  variety  of  personnel  talents>  differential  selec¬ 
tion  techniques  based  on  a  full  understanding  of  job  and 
task  requirements  are  required.  Thus,  the  AFHRL  R§D  pro¬ 
gram  in  personnel  utilization  focuses  on  developing  improved 
techniques  for  personnel  selection  and  job  assignment. 
Studies  conducted  include  development  and  test  of  new 
selection  devices ;  specification  of  job  aptitude  require¬ 
ments;  assessment  of  job  performance  and  job  satisfaction; 
and  development  of  models  of  the  Air  Force  manpower  and 
personnel  system.  These  requirements  are  fulfilled  at  the 
request  of  the  Deputy  Chief  of  Staff  for  Personnel ,  other 
Air  Staff  offices,  and  the  major  air  commands  (MAJCOMs) , 
through  a  formal  request  procedure  specified  in  AFR  80-51. 
Approximately  501  of  all  AFHRL  R8D  is  conducted  in  response 
to  such  formal  requests  while  the  remainder  is  conducted  in 
response  to  tqchnplogy  needs,  regulations,  letter  requests, 
and  work  unit  proposals  submitted  by  AFHRL  scientists. 

Recent  products  in  the  area  of  personnel  utilization 
include:  (1)  a  person-job  match  algorithm  which  matches 

enlistee  talents  with  available  Air  Force  jobs;  (2)  UPT 
selection  procedures  which  could  reduce  selection  costs; 

(3)  specification  of  minimum  aptitude  requirements  for  Air 
Force  jobs;  and  (4)  development  of  a  Vocational  Interest 
Inventory  (VOICE)  which  when  used  with  other  selection 
instruments  will  increase  the  probability  of  enlisted  job 
satisfaction  and  retention. 

Technical  Training  Technology.  The  Laboratory  thrust 
in  technical  training  technology  nas  focused  on  developing 
advanced  instructional  technologies  for  use  in  technical 
training  courses.  The  technologies  developed  are  computer 
based  and  managed,  self-paced,  multi-media,  and  learner 
centered.  They  were  developed  for  initial  use  at  the  ATC 
Lowry  Technical  Training  Center  and  have  been  implemented 
in  four  Lowry  technical  training  courses  with  an  associated 
cost  savings  of  over  $6  million  dollars. 

While  R8D  on  computer  based  instructional  technology 
continues  at  the  Technical  Training  Division,  a  new  emphasis 
is  being  placed  on  maintenance  simulation  techniques.  A 
first  step  has  been  the  acquisition  and  evaluation  of  a 
maintenance  training  simulator  which  can  be  used  in  place 
of  the  actual  flight  control  and  avionics  test  station  for 
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the  F-111  aircraft.  Advantages  of  using  maintenance  simu¬ 
lators  in  technical  training  parallel  benefits  already 
gained  using  flight  simulators.  They  include  reduced  cost 
of  acquisition,  reduced  life  cycle  costs,  and  improved 
training  capability,  such  as  the  capability  to  insert 
faults  into  the  simulator  for  student  isolation.  These 
training  exercises  can  be  conducted  without  exposing  the 
student  to  the  dangerous  voltages  which  exist  in  the  actual 
equipment. 

.  Plying  Training  Technology.  The  current  emphasis  in 
flying  training  technology  is  to  improve  the  utilization 
of  simulators.  AFHRL  also  develops  simulator  design  improve 
ments  when  theproposed  design  is  expected  to  result  in  an 
improved  training  capability.  Current  design  efforts  will 
improve:^  (a)  visual  displays  and  G-cueing  techniques  for 
air-to-air  and  air-to-ground  simulation,  and  (b)  sensor 
(LLLTV ,  IR)  simulation  displays  for  aircrew  training. 

The  evaluation  of  new  simulator  design  concepts  is 
conducted  using  the  Advanced  Simulator  for  Pilot  Training 
(ASPT)  located  at  the  AFHRL  Flying  Training  Division  at 
Williams  AFB.  When  first  installed  in  1975,  the  ASPT  con¬ 
sisted  of  two  T-37  cockpits,  a  6°  of  freedom  motion  plat¬ 
form,  and  a  wraparound  computer  image  generated  display. 
Considerable  R§D  has  been  completed  in  support  of  Air 
Training  Command  (ATC)  and,  more  recently,  in  support  of 
Tactical  Air  Command  (TAC) .  Major  projects  completed  for 
ATC  include  development  of  a  syllabus  for  the  ATC  Instru¬ 
ment  Flight  Simulator  and  specification  of  simulator  motion 
requirements  for  undergraduate  pilot  training.  Current 
projects  in  support  of  TAC  are  A-10  and  F- 16  transition 
training/research  programs  for  which  the  ASPT  has  been 
modified,  and  development  of  a  skills  maintenance  program 
for  TAC  pilots.  Studies  in  air-to-air  combat  techniques 
are  also  being  supported  by  AFHRL  at  Luke  AFB  using  the 
simulator  for  air-to-air  combat. 

Major  FY79  thrusts  in  flying  training  R§D  are  the 
design  and  development  of  an  improved  air-to-air/air-to- 
ground  tactical  air  combat  display,  development  of  auto¬ 
mated  performance  measures  for  aircraft  and  simulators, 
design  and  evaluation  of  low-cost  G-cueing  devices,  and 
specification  of  methods  for  measuring  and  maintaining 
aircrew  skills . 
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For  the  Laboratory  in  general ,  the  maj  or  thrusts  for 
FY79  and  80  are:  computer  based  selection  testing,  apti¬ 
tude  requirements  for  AF  jobs,  maintenance  of  piloting 
skills,  design  specifications  for  maintenance  simulators, 
specifications  for  advanced  air  combat  displays,  and  improved 
force  and  motion  cueing  techniques. 

Aerospace  Medical  Division* 

the  Aerospace  Medical  Division  has  important  mission 
responsibilities  in  medical  education  and  medical  cqre; 
however,  today  I  will  discuss  only  the  AMD  R$D  mission. 

That  mission  is  accomplished  by  the  Sthool  of  Aerospace 
Medicine  (SAM)  and  the  Aerospace  Medical  Research  Labora¬ 
tory  (AMRL)  under  the  direction  of  the  AMD  Directorate  of 
Research  and  Development  located  at  HQ  AMD,  Brooks  AFB, 

Texas. 

The  AMD  R8D  mission,  referred  to  as  Aerospace  Bio¬ 
technology,  is  to  conduct  RDT8E  in  support  of  crew-centered 
design  of  weapon  systems  and  in  support  of  the  operation  of 
manned  Air  Force  weapon  systems.  To  accomplish  that 
mission,  AMD  is  authorized  995  R^D  personnel  and  has  several 
unique  research  facilities.  These  facilities  include  bio¬ 
dynamic  simulators,  toxic  chemical  exposure  chambers,  elec¬ 
tromagnetic  radiation  sources,  and  systems  for  evaluating 
man- in- the- loop. 

Aerospace  biotechnology  is  organized  into  seven 
Technology  Planning  Objectives.  Each  of  the  seven  R8D 
divisions  of  SAM  and  AMRL  is  responsible  for  a  specific 
Technology  Planning  Objective  and  its  supporting  project. 
Major  objectives  address  the  following  technical  areas: 

1)  Environmental  Toxicology,  2)  Radiation  Hazards,  3)  Mechan¬ 
ical  Forces,  4)  Man-Machine  Integration,  5)  Aerospace  Medi¬ 
cine,  6)  Crew  Technology,  and  7)  Manned  Weapon  Systems 
Effectiveness. 

Biotechnology  products  include  human  criteria  for 
system  development  and  acquisition,  and  human  standards  for 
crew  safety  and  environmental  quality.  The  user  incorporates 
these  products  into  design  analyses,  concept  demonstrations, 
and  operational  planning.  The  return  to  the  Air  Force  is 


*Significant  contributions  were  made  to  this  section  by 
personnel  of  AMD/RD. 
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improved  readiness,  a  reduction  in  life  cycle  costs,  and 
human  benefits.  Biotechnology  products  have  a  wide  range 
of  customers:  the  AFSC  product  divisions  and  hardware 
laboratories,  the  Air  Force  Surgeon  General,  and  the  major 
air  commands . 

There  are  three  major  thrusts  in  the  Biotechnology 
area:  (a)  Manned  Weapons  Effectiveness,  (b)  Capability 

Enhancement,  and  (c)  Safety  and  Environmental  Quality. 

I  will  describe  the  objectives  of  each. 

Manned  Weapons  Effectiveness.  The  objectives  of  the 
Manned  Weapons  Effectiveness  research  program  are  princi¬ 
pally  directed  toward  assessment  of  blue  (friendly)  system 
vs  red  (threat)  system  capability  and  the  development  of 
human  countermeasures.  Blue  system  technology  is  focused 
on  1)  developing  a  fundamental  understanding  of  man's 
perceptual/motor  and  cognitive  performance;  2)  translating 
fundamental  information  on  human  behavior  into  design 
standards  and  criteria;  3)  developing  from  this  human  per¬ 
formance  data  base  specific  hardware  and  software  products 
that  enhance  man's  ability  to  conduct  air  operations; 

4)  accomplishing  design  simulations  of  man-machine  systems 
to  provide  specific  software  and  hardware  sblutions  for 
strategic,  tactical,  and  command  and  control  weapon  systems; 
and  5)  providing  direct  consultative  support  to  weapon 
systems  undergoing  development  by  the  various  divisions  of 
the  Air  Force  Systems  Command. 

Red  system  technology  concentrates  on:  1)  the  quanti¬ 
fication  Of  human  operator  contributions  to  foreign  threat 
weapon  system  effectiveness,  and  2)  the  development  of 
techniques  to  reduce  or  nullify  the  effectiveness  of 
threat  weapon  systems  operators.  It  is  generally  accepted 
that  a  valid  description  of  the  crew  subsystem  is  the  crux 
of  viable  weapon  system  effectiveness  evaluation.  Sensi¬ 
tivity  evaluations  of  large-scale  engagements  using  analyti¬ 
cal  techniques  are  conducted  to  determine  crew  impact  on 
the  total  system  and  the  level  of  confidence  which  may  be 
placed  on  system  effectiveness  predictions.  Increasing 
requirements  for  analysis  of  Air  Force  weapon  system  capa¬ 
bility  and  survivability  conflict  with  the  high  costs  of 
flight  and  field  tests.  This  situation  underscores  the  need 
to  describe  crew  subsystems  in  the  same  terms  as  other 
weapon  system  components  so  that  automated  analyses  can  be 
accomplished  at  much  lower  cost. 


1645 


Capability  Enhancement.  The  Capability  Enhancement 
thrust  encompasses  crew  protection,  crew  readiness,  and 
fitness  to  fly.  Diminishing  availability,  increased  cost, 
and  more  demanding  mission  requiremeiits  dictate  the  need 
to  employ  air  and  ground  crews  with  maximum  efficiency. 

The  requirement  for  weapon  systems  to  operate  in  an 
environment  of  expanding  threats  to  man's  capability  has 
underscored  the  need  for  advanced  life  support  equipment 
to  replace  currently  available  life  support  assemblies  that 
are  increasingly  costly.  Thus,  in  the  area  of  crew  pro¬ 
tection,  AMD  is  developing  design  Criteria  for  protective 
equipment  for  the  aircrew,  and  environmental  control  and 
emergency  escape  systems  for  the  aircraft.  Crew  readiness 
research  focuses  on  establishing  criteria  for  crew  ratio 
requirements,  duty  schedules,  and  sortie  surge  which  may 
be  encountered  in  rapid  overseas  deployment.  Possible 
aircrew  fatigue  is  the  central  concern  in  establishing 
these  criteria.  In  the  fitness  to  fly  area,  AMD  has  the 
mission  of  establishing  criteria  for  medical  selection  and 
retention  of  UPT  and  UNT  applicants.  Qualified  pilots  are 
also  evaluated  when  medical  problems  arise.  AMD  is  also 
involved  in  developing  patient  monitoring  equipment  for 
air  evacuation  aircraft. 

Safety  and  Environmental  Quality.  The  objectives  of 
the  Safety  and  Environmental  Quality  research  program 
center  around  the  prediction,  assessment,  and  specification 
of  safe  exposure  limits  for  airbase  noise,  electromagnetic 
radiation  and  military  chemicals.  Noise ^research  is  pro¬ 
viding  criteria  for  compatible  land  use  in  the  vicinity  of 
the  airbase  as  well  as  safety  standards  to  protect  air  and 
ground  crew  personnel  in  Air  Force  noise  environments.  The 
objectives  of  the  radiation  bioeffects  research  program  are 
to  assess  hazards,  quantify  acute  and  delayed  biologic 
effects  of  nonionizing  laser /maser,  nuclear  flash,  radio¬ 
frequency,  and  ionizing  radiation  on  man.  The  objectives 
of  the  toxicology  research  programs  are  to  protect , ^maintain 
and  enhance  the  performance  of  Air  Force  personnel  in  poten¬ 
tially  hazardous  chemical  environments  associated  with  the 
operation  and  maintenance  of  aircraft  and  missile  weapons 
systems.  The  exposure  hazard  covers  the  entire  spectrum  of 
low-level  continuous  or  intermittent  to  high-level  brief 
accidental  or  unavoidable  exposures.  Bioeffects  data  are 
required  to  establish  standards  compatible  with  recent 
Federal  legislation  concerning  occupational  safety  and 
health,  and  control  of  hazardous  materials  in  the  context 
of  both  military  operations  and  environmental  quality. 
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In  summary,  the  major  R§D  thrusts  of  AFHRL  are 
Personnel  Utilization,  Technical  Training  Technology 
and  Flying  Training  Technology.  The  major  R§D  thrusts 
of  AMD  are  Manned  Weapons  Effectiveness,  Capability 
Enhancement,  and  Safety  and  Environmental  Quality. 

Both  organizations  contribute  significantly  to  the  effec¬ 
tiveness  and  readiness  of  Air  Force  operations  through  the 
development  of  new  technologies  and  devices  designed  to 
increase  man's  productivity  and  safety. 
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Human  Resources  in  Naval  Aviation 
Abstract 


The  objectives  of  the  Navy  Training  and  Personnel 
Technology  Program  are  to  achieve  best  use  of  military 
manpower  and  to  make  a  significant  contribution  to  the 
readiness  of  operational  forces.  Problems  dealt  with 
this  program  include  personnel  acguisition,  training, 
manpower  management,  and  the  human/weapons  system  per¬ 
formance  interface. 

A  matter  of  concern  for  the  Training  and  Personnel 
Technology  Program,  as  fbr  all  NaVy  piro^rams,  is  that  of 
cost.  Manpower  and  related  costs  for  the  Department  of 
Defense  represent  the  largest  single  expense/  amounting 
to  almost  60  percent  of  the  total  defense  budget.  Effec¬ 
tive  and  economical  use  of  these  manpower  resources  is  a 
matter  of  top  priority*  At  the  same  time,  it  is  necessary 
to  control  costs  for  military  hardware.  A  single  aircraft 
can  cost  well  over  $20  million.  Full  benefit  from  our 
investment  in  both  manpower  and  hardware  cah  only  be 
realized,  however,  if  proper  attention  is  given  to  the 
personnel  factors  in  Navy  systems,  including  training, 
'Utilization,  and  maintenance.  This  paper  reviews  use  of 
hvimah  resources  in  the  Navy,  discusses  some  cost  factors, 
and  describes  significant  trends  and  representative  RDT&E 
efforts.  Particular  note  is  taken  of  advanced  technologies 
being  used  and  their  potential  for  improving  cost-benefit 
ratios  for  new  Navy  weapon  systems. 


■- Introduction 

The  advanced  technologies  now  being  applied  to  the 
design  of  weapons  systems  are  resulting  in  dramatic  in¬ 
creases  in  military  capabilities  arid  effectiveness. 

However,  With  each  technological  stride  there  is  a  cost, 
often  little-noted  or  anticipated,  in  the  roles,  tasks,  and 
commitments  of  personnel.  Someone  must  decide  how  tbe  new 
system  will  be  used;  someone  must  learn  to  operate  it; 
someone  must  maintain  it.  However  simply  stated  here,  these 
issues  are  not  easily  resolved  in  the  real  world.  The 
application  of  hew  technologies  often  results  in  systems 
which,  although  impressive  from  the  strict  military  view, 
are  exceedingly  difficult  to  understand,  to  operate,  and  to 
maintain.  The  resulting  mismatch  between. the  hardware  and 
personnel  components  of  new  systems  can  add  considerably  to 
system  costs  while  at  the  Same  time  reducing  effectiveness. 
One  important  way  of  improving  the  cost/effectiveness  ratio 
for  new  systems  is  through  advances  in  the  technology  with 
which  we  Use  our  human  resources. 

The  theme  of  this  conference  concerns  applications  of 
new  technologies  in  military  systems  at  affordable  costs. 
Proper  resolution  of  "operator"  issues  can  dp  much "to  Ibwer 
total  system  costs.  However,  this  is  a  two-edged  issue. 
While  strivilvg  to  lower  system  costs,  one  must  not  let  the 
personnel  technology  itself  become  too  expensive.  This 
paper  reviews  use  of  hvirnan  resources  in  the  Navy,  discusses 
some  cost  factors,  and  describes  significant  trends  and 
representative  RDT&E  efforts. 

The  term  "Human  Resources"  is  global  and  has  varying 
and  sometimes  misleading  definitions.  For  example^  the  Navy 
now  has  a  Hxjman  Resources  Management  Program  which  concen¬ 
trates  bn  maximizing  the  effectiveness  of  Navy  men  and 
women  by  dealing  with  issues  of  drug  abuse,  alcohol  abuse, 
and  equal  opportunity.  To  reduce  the  confusion  in  termi¬ 
nology.  Congress  agreed  with  DbD's  deeire  to  adopt  the  term 
"Training  and  Personnel  Technology  Programs"  to  cover 
topics  of  concern  in  this  report.  In  the  House  Armed 
Services  Conimittee  report  accompanying  the  FY  1979  DoD 
Authorization  Bill,  it  was  recommended  that  future  budget 
elements  of  this  progreun  be  divided  into  four  categories  as 
shown  in  Exhibit  1.  Specific  program  tasks  were  defined  by 
the  Committee  as  follows : 


Training  Pevicas  and  Siroulators  ■"  All  efforts  to 
design  and  develop  maintenance,  flight,  and  combat 
engagement  simulators.  ^  ^ 

Human  Factors  -  All  efforts  to  develop  techniques, 
procedures ,  and  criteria' for  the  design  of  weapon 
systems  so  that  they  can  be  efficiently  PP®?^^be^ 
and  maintained  by  military  personnel. 


Manpower  and  Personnel  "•  All  efforts  fo  develop  a 
better  understanding  of  manpower  requirements, 
methods  of  recruiting^  incentives  for  retention, 
personnel  management  and  organizational  effectxveneSs. 


Computer-Aided  and  Classroom  Training  -  Ail  efforts 
to  develop  or  adapt  new  techniques  for  training 
personnel.  Examples  include  computer-managed 
training,  lesson  development,  and  handbook  prepa” 


ration. 


Program  Support  and  Facilities 

•  The  funding  for  the  major  areas  in  the  Training  and 
Personnel  Technology  Program,  for  ail  of  DoD,  is  shown  in 
Exhibit  2,  The  figures  presented  in  the  final  col^n,  for 
FY  1978,  are  an  earlier  estimator  with  the  actual  funding 
now  known  to  be  around  $90  million  rather  than  the  $104 
million  shown.  Even  so,  the  figures  show  a  relative  _ 
balance  among  the  four  areas  as  well  as  a  modest  growth 
over  the  three— year  period. 

Each  technology  area  shown  in  Exhibit  i  supports  a 
partieular  aspect  of  military  operations.  For  example, 
"Human  Factors"  concerns  the  design  of  weapon  systems  for 
efficient  utilization  by  human  operators.  Likewise, 
"Training  Devices  and  Simulation"  supports  the  development 
and  utilization  of  training  systems  which  serve  both  to 
improve  the  quality  of  training  and  to  reduce  the  need  to 
use  operational  systems  as  trainers. 

In  using  a  training  device  such  as  a  flight  simulator 
there  are^ two  costs  to  consider.  The  first  obviously  is 
the  cost^^f  acquiring  the  device.  This  cost  is  not  small. 


^DoD  has  Substituted  the  term  "Education  and  Training"  to 
designate  tasks  in  this  category. 
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Over  the  past  five  years,  the  Department  of  Defense  has 
Ihvested  about  $1.3  billion  in  the  development,  acquisition, 
and  modernization  of  flight  simulators.  In  at  least  one 
Instance,  the  cost  of  a  single  flight  simulator  has  ap¬ 
proached  $25  million. 

The  second  cost  is  that  of  developing  a  technology  for 
efficient  use  of  the  simulator.  Here  there  are  many  issues 
such  as  acceptance,  fidelity,  scheduling,  measurement,  and 
others.  Suffice  it  to  say  that  it  is  most  important  to  see 
that  our  $1.3  billion  investment  produces  the  proper  return 
in  terms  of  lower  flight-hour  costs  and  increased  combat 
readiness. 

It  is  of  interest  now  to  compare  the  costs  for  the 
acquisition  of  flight  simulators  with  the  funding  for  the 
training  and  personnel  technology  area  designed  to  support 
efficient  use  of  these  devices.  Exhibit  3  shows  the  funding 
for  procurement  of  Navy  and  Marine  Corps  flight  simulators 
from  FY  1976  through  FY  1980.  This  is  compared  with  funding 
for  the  training  technology  program  which  defines  effective 
design  and  utilization  of  these  simulators.  This  table 
shows  that  simulator  acquisition  costs  rose  rather  dramati¬ 
cally  until  1978,  at  which  point  they  have  stabilized  at 
around  $125  million  per  year.  On  the  other  hand,  funding 
for  the  R&D  technology  to  support  use  of  these  simulators, 
while  showing  a  modest  growth,  has  in  fact  decreased  in 
relation  to  simulator  costs.  In  1976,  supporting  tech¬ 
nology  costs  were  about  four  percent  of  acquisition  costs. 

At  present,  and  for  the  next  two  years,  the  R&D  funding  has 
dropped  to  two  percent.  This  relationship  is  presented 
graphically  in  Exhibit  4. 

The  relative  decline  in  R&D  funding  for  simulator 
support  can  be  seen  through  another  comparison.  Exhibit  5 
shows  the  total  funds  for  all  Navy  efforts  in  the  category 
of  "Exploratory  Development."  This  is  compared  with  the 
total  Navy  and  Marine  Corps  budget  for  the  years  1976 
through  1979.  The  Exploratory  Development  funds  run  at  a 
constant  one  percent  of  the  total  Navy  budget  through  this 
period.  Thus,  whereas  the  overall  research  funding  is  a 
fixed  ratio  to  total  Navy  costs,  that  part  which  supports 
efficient  use  of  simulators  shows  a  decreasing  ratio  when 
compared  with  simulator  acquisition  costs. 

The  funding  relationships  just  described  have  con¬ 
siderable  implications  for  our  R&D  program.  First,  bearing 
in  mind  that  there  is  a  1974  DoD  mandate  to  reduce  military 
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flying  hours  by  25  percent  by  1981,  there  is  an  ever-growing 
need  to  develop  optimized  techniques  for  using  simulatprs. 

The  increasing  supply  of  these  training  devices  must  be  used 
to  best  advantage  as  a  substitute  for  flight  time  and  in 
such  a  manner  as  to  improve  combat  readiness.  Second,  man¬ 
agers  of  the  R&D  technology  effort  are  forced  to  work  within 
a  budget  which  has  not  shown  the  same  growth  as  that  noted 
for  simulators  themselves.  This  review  will  describe  some 
of  the  ways  in  which  we  are  meeting  the  challenge  of  pro¬ 
viding  proper  personnel  support  while  at  the  same  time 
seeing  that  our  technology  development  costs  remain  at  an 
affordable  level. 

Exhibit  6  shows  the  principal  Navy  organizations 
engaged  in  RDT&E  efforts  dealing  with  the  four  Training  and 
Personnel  Technology  elements.  Depending  upon  the  particular 
problem,  any  one  of  these  facilities  can  work  in  direct 
support  of  a  NAVAIR  project  manager. 

Now  that  we  have  defined  the  scope  of  hviman  resources, 
discussed  cost  constraints,  and  shown  where  the  RDT&E 
program  is  conducted,  we  will  describe  issues,  trends,  ahd 
current  efforts  in  this  field. 


Key  Issues 

There  are  a  number  of  key  issues,  shown  in  Exhibit  7, 
which  human  resources  programs  must  face.  The  first  of 
these  is  the  increasing  number  of  new  weapons ,  command  and 
control,  and  surveillance  systems  now  being  introduced  into 
the  Fleet  and  the  complexity  of  these  systems.  The  niambers 
alone  are  most  impressive.  In  the  recently-completed 
Project  HARDMAN  report,  dealing  with  military  manpower 
versus  hardware  procurement,  it  was  noted  that  there  are 
presently  in  procurement  some  700  different  Navy  projects 
involving  approximately  $90  billion. 

A  good  example  of  the  complexity  of  these  new  systems 
can  be  found  in  V/STOL  aircraft  technology.  In  a  review  of 
the  hviman  factors  problems  presented  by  these  aircraft  con¬ 
ducted  by  the  Naval  Air  Systems  Command,  it  was  concluded 
that; 

Demands  in  V/STOL  of  necessity  ■place  a  higher 
priority  upon  stability,  control  and  display 
relationships  than  do  requirements  for  conventional 
flight.  Current  military  specifications  and 
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standards  are  geared  to  the  latter  requirements, 
and  fall  far  short  of  meeting  the  umque  V/STOL 
challenge.  Advanced  technology  programs  .  .  . 

afford  the  required  hardware  and  software  capab^l^t^es 
to  Batiafy  many  of  the  unique  v/STOL  requirements, 
but  oonaiderable  effort  will  he  required  to  define 
and  refine  the  information,  control  and  dt-sp  Lay 
concepts  and  requirements  for  implementatvon. 

Another  example  of  complexity  issues  can  be 
the  panel  layout  of  new  aircraft.  The  F-4  fighter  had  _  ^ 
1,750  square  inches  of  space  available  for  cockpit  <^^spl  y. 
In  the  F-18,  this  has  been  reduced  to  850  square  inches,  or 
Sproximately  one^half.  At  the  same  time,  the  introduction 
of  new  target  acquisition  systems,  such  as  ^ 

sidera±>ly  increased  the  information  which  must  be  displayed. 
This  obviously  means:  more  information  P?t_display  system  and 
multipurpose  displays.  The  additional  information,  co^med 
with  sequential  presentation  modes,  can  greatly  . 

aircrew^ workload  and  result  in  poorer  rather  than  in  better 
overall  system  performance. 

A  second  issue  of  importance  is  that 
tenance.  Admiral  Michaelis  recently  observed  that  ^he 
contribution  of  maintenance  logistics  ^ 

and  support  costs  is  disproportionately  high.  With^new  ^ 
aircraft  such  as  the  F-18,  considerable ^ a ttention^is  being 
given  to  increased  reliability  through 

for  maintainability  and  through  new  concepts  to  survey  and 

control  maintenance  activities.  ^^®^®  desian 

computer-based  management  information  network  during  ^®sign 
and  development,  and  Continuing  through  Fleet  introductio  , 
to  track  all  system  failures.  Admiral  Michaelis  predicts 
Sar“creisea  reliability,  along  with  improved  narntain- 
ability,  may  reduce  the  requirement  for  v  a 

personal  by  as  much  as  20  percent  over  that  for  the  F-4 
aircraft.  However,  until  such  maintenance  improvement  i 
actually  experienced,  improvements  in  ^e  training  an 
support  of  maintenance  personnel  will  be  a  top  priority 

item. 

A  strong  emphasis  in  maintenance  training  now  is 
toward  simulation.  Maintenance  simulators  have  more  flexi¬ 
bility  in  use,  expedite  the  training  process,  and  most 
important,  are  considerably  less 
equipment  procured,  as  is  often  the  case,  solely 
tLance  trLning  purposes.  The  development  and  production 
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f  1  is  a  new  market,  one  which  we 

anticipate  will  grow  quite  rapidly  in  the  next  several 

suDDo^t"‘fii'^ni^”^  •  PJ^oblem  of  providing  proper  personnel 

suDDlv  J  the  fact  that  the  basic 

dl^oS^L  changing.  Part  of  this  simply  is  a 

reSui^i^a^^^?^  ?*  i  1977,  the  Navy  did  not  meet  its 

«w^  ^  i  116,000  recruits,  falling 

for^hirth  while  achieving  a  recruiting  rate 

le«  ihan  SSliedt''  ■"*  Sraduates  of  3.4  percent  age  points 

1-r-a-ir.f”^  Pflot  training ,  the  number  of  candidates  entering 
training  is  adequate  but  retention  after  the  first  tour  is 
deteriorating.  Prom  FY  1978  to  FY  1979  a  drop  in  retention 

percent  is  projected.  Since  a  certain 
•  .v®  pilots  will  leave  the  service  no  matter  what 

J  ^  introduced,  it  is  up  to  us  to  see 

pilots  achieve  maximum  operational  capability 
while  they  are  in  service.  , 

^  Another  issue  to  be  addressed  is  the  relationship 
between^ the  human  resources  R&D  community  and  the  Fleet. 
Although  in  concept  all  R&D  activities  in  some  manner  should 
^  support  Fleet  readiness,  the  proper  balance  to  be 
achieved  among  short-,  mid-,  and  long-term  payoffs  has  not 
always  been  clear  to  either  researcher  or  user.  Recently, 
however,  there  has  been  a  concerted  effort  to  improve 

For  example,  in 

June  1977,  the  Navy  sponsored  a  three  and  one-half  day 
symposium  dealing  with  the  utilization  of  people-related 
research.  Key  DoD  decision  makers,  technical  advisory 
groups,  and  potential  users  of  military  research  were 
present  at  this  symposium.  Other  efforts  such  as  this  are 
planned  in  an  ongoing  program  to  improve  the  exchange  of 
requirements  information  On  one  hand  and  research  findings 
on  the  other. 

r.::.,.  The  final  issue  is  that  of  developing  ways  in  which  the 
pay-off  from  human  resources"  RDT&E  activities  can  be 
clearly  demonstrated.  There  is  a  need  to  evaluate  these 
research  efforts  in  objective  measures  describing  improved 
system  performance  and/or  decreased  costs  for  system  de¬ 
velopment  and  operation.  Evaluations  such  as  this  are 

First,  we  must  be  able  to  demonstrate, 
particularly  to  those  responsible  for  budget  allocations. 
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that  the  human  resources  technology,  easily  justified 
rationally,  can  be  just  as  easily  justified  in  terms  of 
objective  value  to  the  Navy.  Second,  evaliiation  data  are 
important  to  research  managers  as  a  basis  for ^ continuing 
improvements  in  our  human  resources  technologies. 


Trends 

Within  the  past  five  years,  there  have  been  a  number 

of  significant  changes  and  reorganizations  within  the  Navy 

human  resources  community.  Principally,  there  is  more 
cohesion  in  the  effort  today  so  that  the  overall  power  of 
the  program  can  be  brought  to  bear  on  specific  Navy  problems 
For  example,  in  support  of  the  F-18  aircraft  program,  there 
are  a  number  of  separate  R&D  efforts  underway  at  different 
facilities.  However,  these  are  not  independent  activities 
conducted  along  specific  lines  of  interest^ to  the, ^principal 
investigator,  as  might  have  been  the  case  in  earlier  days. 
They  now  are  part  of  a  coordinated  effort  in  which  Fleet 
reguirements  are  processed  through  the  Naval  Air  Systems 
Command  and  its  lead  laboratory,  the  Naval  Air  Development 
Center.  The  activities  of  the  different  laboratories  thus 
are  focused  toward  the  single  objective  of  providing  greatly 
improved  personnel  support  for  the  F-18. 

Exhibit  8  shows  some  specific  trends  now  underway  in 
human  resources  RDT&E  activities.  The  first  of  these  is 
the  incorporation  of  advanced  technologies  into  human  re¬ 
sources  programs,  just  as  these  technologies  underlie  the 
development  of  actual  weapon  systems.  For  instance,  a  new 
program  at  the  Naval  Training  Eguipment  Center  is  incor¬ 
porating  the  use  of  minicomputer  and  microprocessor 
technology  into  the  development  of  low-cost,  part- task  ^ 
training  devices  which  may  be  acquired  in  quantity.  Such 
devices  should  improve  the  quality  of  pilot  training  as 
well  as  reduce  the  requirement  for  veiry  expensive  full- 
mission  simulators.  In  this  program,  the  pilot  trainee 
sits  in  front  of  a  small  console  containing  a  minicomputer 
graphics  display.  Instruments  are  shown  in  the  lower  half 
of  the  display  with  an  out-of-the-window  view  in  the  top 
half .  There  are  appropriate  controls  for  stick ,  throttle , 
radio,  and  weapons  control.  The  minicomputer  performs  all 
simulation  computations ,  as  well  as  handling  scoring  and 
problem  set-up.  Catalogs  of  various  aircraft  dynamics  and 
exercises  are  available  on  floppy  discs*  Sample  problems 
include  night  carrier  landing ,  approach  and  landing  at • a 
bingo  field,  and  real-time  gun-sight  operations. 
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Interchangeable  control  modules  allow  representation  of 
various  Naval  Flight  Officer  functions,  such  ais  those  of 
the  Radar  Intercept  Officer.  The  first  of  these!  devices  is 
scheduled  for  evaluation  at  a  Fleet  squadron  in  FY  1979  to 
assess  the  impact  of  this  kind  of  training  oh  night  landing 
performance  as  measured  during  pre-deployment  training. 

The  second  trend  involves  a  much  greater  use  of  com¬ 
puters  and  computer-modeling.  The  Naval  Air  Development 
Center  has  had  a  vigorous  program  underway  in  this  respect. 
They  have  developed  the  Computer-Aided  Function  Allocation 
and  Evaluation  System,  with  a  nuinber  of  subprograms  which 
greatly  improve  pur  capability  to  deal  with  operator  issues 
during  aircraft  design,  development,  and  evaluation.  These 
models  include  one  called  the  Human  Operator  Simulator. 
Through  use  of  this  model,  and  other  subroutines,  it  is 
possible  to  deal  with  problems  of  function  allocation, 
workload  assessment,  internal  design,  and  control/display 
groupings.  The  model  for  operator  simulation  currently  is 
being  adapted  for  cost-effectiveness  estimation.  Through 
simulation  of  the  complete  hardware/software/operator  system, 
it  will  provide  predictions  of  effectiveness  attributable  to 
proposed  avionics  changes  and  will  allow  us  to  determine, 
prior  to  implementation  of  these  changes,  which  of  several 
configurations  produces  the  greatest  increase  in  total 
system  performance,  with  the  role  of  the  human  opera tor 
fully  taken  into  account  in  each  configuration. 

A  third  trend  concerns  the  movement  toward  greater 
exchange  of  data  within  human  resources  disciplines.  This 
serves  to  make  the  entire  human  resources  effort  more 
cost-effective  and  makes  outputs  more  timely.  For  example, 
computer  models  for  different  human  engineering  tasks 
produce  data  files  which  have  immediate  utility  for  training. 
In  the  case  of  function  allocation  and  workload  prediction 
models,  data  can  either  be  used  directly  for  training 
analyses  6r  can  serve  as  the  beginning  point  for  necessary 
design  modifications.  This  being  the  case,  we  hope  to 
develop  means  which  will  allow  automatic  hand-off  of  common 
data  between  such  facilities  as  the  Naval  Air  Development 
Center  and  the  Naval  Training  Equipment  Center.  A  computer¬ 
ized  link  would  mean  that,  as  some  change  is  made  in  system 
design,  we  could  rapidly  assess  its  implications  for  change 
in  training  system  planning.  It  also  means  that  the  Navy 
would  pay  Only  once  for  the  development  of  a  data  base. 

This  is  not  always  the  case  now. 
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The  final  trend  of  importance  is  the  increased  involve¬ 
ment  of  the  Fleet  in  the  development  of  training  systems. 

The  interaction  and  cooperation  between  Fleet  personnel  and 
training  personnel  is  considerable  and  occurs  at  ^ii  stages 
in  the  development  of  training  hardware.  The  LAMPS  system 
provides  a  good  example  of  how  this  cooperation  works,  as 
shown  in  Exhibit  9.  At  the  present  time,  a  training  com¬ 
mittee  has  been  established  to  review  requirements  and  to 
develop  guidelines  for  production  and  use  of  training 
equipment.  This  training  committee  includes  representatives 
from  the  Chief  of  Naval  Operations,  the  Naval  Air  Systems 
Command,  the  Chief  of  Naval  Education  and  Training,  Atlantic 
and  Pacific  Fleet  Commands,  and  a  nximber  of  the  Navy  labo¬ 
ratories,  with  the  Naval  Training  Equipment  Center  serving 

as  Program  Manager.  There  also  is  a  Fleet  training  team 
located  at  the  site  of  the  weapon  system  prime  contractor 
to  coordinate  and  review  the  acquisition  of  training  har  - 
ware.  Subsequently ,  as  the  weapon  system  becomes  operational, 
there  will  be  a  Fleet  project  team  to  review  the  desip  of 
simulation  equipment  and  a  Fleet  implementation  team  to  ai 
the  integration  of  simulation  equipment  into  the  training 
schedule  and  to  develop  evaluation  programs. 


Current  Activities 

A  brief  review  of  ongoing  projects  will  show  the  kind 
of  effort  now  underway  in  each  of  the  four  categories 
described  earlier  in  Exhibit  1.  These  projects  also  provide 
examples  of  the  way  h\aman  resources  efforts  are  helping  tne 
Navy  achieve  the  "affordability"  goal. 


Training  Devices  and  Simulators 


Due  both  to  the  needs  for  cost  reduction  and  energy 

conservation,  there  is  considerable  interest  in  the ^develop¬ 
ment  of  new  and  improved  simulation  equipment.  In  fact. 
Congress  has  mandated  that  the  services  will  use  simulators 
in  support  of  flight  training  to  the  fullest  extent  possible. 
An  obvious  impetus  is  the  dramatic  reduction  in  costs  which 
can  be  achieved,  as  shown  in  Exhibit  10.  The  data  on  which 
these  summary  figures  are  based  come  from  Air  Force,  Army, 
Navy,  and  commercial  sources.  Since  many  assumptions  were 
made  in  the  use  of  this  information,  the  costs  shown  repre¬ 
sent  an  estimate  only,  although  one  would  expect  that  this 
estimate  is  quite  close.  In  any  event,  it  is  reasonable^to 
anticipate  a  ten-to-one  reduction  in  costs  when  efficient 
simulators  are  used  to  replace  aircraft  flight  hours. 
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A  major  effort  now  underway  at  the  Naval  Training 
Equipment  Center  concerns  the  development  of  an  advanced 
R&D  facility,  known  as  the  Aviation  Wide  Angle  Visual 
System  (AWAVS) ,  shown  in  Exhibit  11.^  Use  of  this  facility 
will  allow  us  to  examine  the  most  affordable  and  flexible 
ways  of  presenting  visual  information  to  aircrewmen  in 
simulators.  In  AWAVS,  there  are  two  independent  television 
projectors  and  a  wide  angle  display  screen.  The  wide  angle 
projector  displays  the  background  scene  while  the  narrow 
angle  target  projector  provides  high  resolution  target 
images.  Projectors  are  fed  from  a  flexible-image  storage 
system  which  includes  three-dimensional  models,  two- 
dimensional  film  and  computer  image-generation  equipment. 
Upon  completion,  the  AWAVS  system  will  represent  a  greater 
advance  in  display  technology  than  any  other  simulator 
currently  available.  It  will  provide  features  of  motion, 
configuration  change  flexibility,  multiple  image  generation, 
visual  streaming  or  ground  growth  cues,  plus  other  capa¬ 
bilities.  Ultimately  this  program  will  be  capable  of 
portraying  missions  for  such  advanced  systems  as  V/STOL 
aircraft. 


Human  Factors 

Most  air-to-ground  weapons  currently  in  use  require 
that  the  aircrew  make  a  visual  acquisition  of  the  target 
before  the  weapon  can  be  employed.  However,  in  the  eval¬ 
uation  of  different  weapons,  such  as  bombs,  guns,  rockets, 
and  guided  missiles,  effectiveness  generally  is  based  on 
delivery  accuracy  and  target  destruction.  The  probability 
of  finding  the  target  in  time  to  attach  and  launch  the 
weapon  usually  is  ignored. 

The  Naval  Weapons  Center  has  developed  a  method  for 
computing  the  probability  of  locating  a  ground  target 
visually  and  then  launching  a  weapon  against  it.  Major 
factors  used  in  the  computations  are  target  acquisition  per¬ 
formance,  aircraft  maneuvering  requirements,  terrain  masking 
visibility,  and  weapon  operating  time.  Estimates  of  these 
factors  are  based  on  real-world  data  whenever  possible,  as 
opposed  to  mathematical  modeling.  Exhibit  12  shows  the  kind 
of  acquisition  scene  which  can  be  dealt  with  by  this  method, 
in  this  case  anti-tank  missile  launchers  sited  beside  a  road 

The  results  of  the  NWC  study  show  that  the  probability 
of  releasing  or  launching  a  weapon  on  a  target  is  quite  low 
in  many  situations.  The  method  itself  can  be  incorporated 


as  one  part  of  the  evaluation  of  a  new  air-to-ground  wb'apon. 
It  also  can  be  used  to  provide  ah  indicatiori  of  the  relative 
effectiveness  of  visual  search  versus  eleetronic  Search. 


Manpower  and  Personnel 

The  development  of  new  weapon  systems,  Snd  their  Sub-  ' 
sequent  manning,  requires  the  forecasting  with  some  precision 
of  future  personnel  levels  and  requisite  skills.  Exhibit  13 
shows  ^at  after  1980  the  personnel  supply  will  be  in  gradual 
reduction  for  some  time.  Under  theSe  conditions,  the  aegui- 
sitioh  and  appropriate  use  of  personnel  becomes  of  even 
greater  importance  than  it  is  today. 

The  Navy  Personnel  Research  and  Development  Center  has 
several  studies  and  modeling  efforts  underway  to  investigate 
the  long  range  supply  and  appropriate  utilization  of  man¬ 
power.  Results  of  these  studies  will  be  important  for  the 
design  and  staffing  of  new  WeapCns  and  support  systems, 
particularly  those  which  are  manpower  intensive.  In  addi¬ 
tion,  these  studies  define  quaiitative  standards  for 
personnel  in  terms  of  weapon  system  demands. 

The  Naval  Aerospace  Medical  Research  Laboratory ^ 
working  with  the  Naval  Training  Equipment  Center,  is" 
attempting  to  improve  the  selection  process  through  use  of 
synthetic  task  devices  in  selection.  These  devices,  which 
in  a  limited  way  represent  the  operational  world,  should 
tap  basic  capacities  not  measured  with  more  conventional 
tests.  The  result  should  be  the  selection  of  individuals 
more  likely  to  succeed  at  the  job  Under  consideratioh. 

This  will  improve  the  Navy  manpower  situation  and  may  also 
result  in  better  retention  rates. 


Computer-Aided  and  Classroom 
Training  '  - 

A  continuing  Navy  concern  is  for  procedures  which 
will  lead  to  improved  aircrew  survivability,  particularly 
if  the  improvements  Can  be  achieved  at  an  affordable  coSt. 
The  Naval  Air  Systems  Command  recently  undertook  a  study 
of  special  presentation  formats  in  ah  aircraft  NATOPS 
Manual  as  one  way  of  possibly  deducing  the  fatalities  and 
injuries  incurred  during  ejection  from  a  disabled  aircraft. 
This  project  also  received  support  from  the  Navy  Technical 
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Presentation  Program  (NTIPP)  at  the  Navy  Ship  Research  and 
Development  Center  because  of  its  interest  in  the  effects 
of  techhical  manual  format  on  the  performance  of  time- 
•criticalr  hazardous  procedures. 

An  experiment  was  conducted  during  which  the  T-2  NATOPS 
ejection  procedure  presentations  were  redesigned  employing 
state-of-the-art  methods.  Two  groups  of  student  naval 
aviators,  each  of  which  had  completed  ejection  training, 
were  selected.  One  group  studied  the  standard  NATOPS  Manual. 
The  other  studied  the  revised  presentation.  Both  groups 
were  tested,  as  shown  in  Exhibit  14.  These  results  indicate 
that  the  new  Write-up  significantly  increased  the  user's 
awareness  of  ejection  seat  limitations  and  of  procedures 
needed  to  operate  the  hardware.  If  this  knowledge  is  in 
tutn  reflected  in  better  use  of  the  seat  and  better  under¬ 
standing  of  the  ejection  envelope,  as  should  be  the  case, 
this  represents  a  low  cost  increase  in  aviation  safety. 

There  is  another  effort  underway  which  offers  con¬ 
siderable  training  benefit  as  well  as  cost  savings.  For 
several  years,  an  advanced  simulator,  the  Night  Carrier 
Landing  Trainer,  has  been  used  by  the  Replacement  Pilot 
Training  Squadron,  NAS  Cecil  Field,  Florida.  This  is  a 
sophisticated  training  simulator  incorporating  motion 
characteristics  and  all  of  the  visual  information  available 
for  a  night  carrier  landing. 

In  a  recently-completed  Study,  the  training  effective¬ 
ness  of  the  Night  Carrier  Landing  trainer,  both  for  day  and 
night  landings,  was  evaluated.  Exhibit  15  shows  that  for 
day  landings,  the  performance  of  new  aviators  who  were 
given  NCLT  training  using  remedial  techniques  based  on  an 
excunination  of  errors  during  the  training  cycle  was  superior 
to  all  groups,  including  A-7  pilots  with  combat  experience. 
For  night  landings,  again  the  novice  aviators  with  remedial 
NCLT  training  were  above  average.  The  most  direct  evalu¬ 
ation,  however,  comes  from  tJie  considerable  improvement  in 
night  landing  performance  of  hew  aviators  with  NCLT  training, 
but  with  no  special  remedial  aspects ,  oyer  that  of  new 
aviators  who  did  not  use  the  Night  Carrier  Landing  Trainer. 
The  training  benefit  of  the  device  alone  is  obvious  from 
these  scores.  However,  again  it  is  seen  that  the  use  of  a 
personalized  remedial  program  in  conjunction  with  the  NCLT 
results  in  still  more  improvement. 
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The  NCLT  results  make  one  compelling  point.  Simply 
acquiring  a  simulator  is  not  enough.  Maximtam  training 
benefit  oitiy  comes  when  the  simulator  is  supported  by  a 
training  technology  that  tells  how  best  fb  use 
case,  adding  a  proper  training  techno  logy  to  an  bhigoirig 
NCLT  program  results  in  an  additional^  perforTnahce'  Impro^ 
ment  which  is  almost  as  large  as  that  fOuttd  when  the  NCiLT 
device  alone  is  incorporated  in  the  carrier  la.nding  traitiihg 
program.  ■ 

Once  proficiency  in  carrier  landings  is  achieved,  it  is 
most  important  Uiat  it  be  maintained,  particularly  if  some 
time  elapses  between  landings.  Any  loss  in  capability  could 
have  disastrous  results.  However,  we  know,  as  seen  in 
Exhibit  16,  that  there  is  a  normal  decay  function  for  this 
type  of  performance  through  time.  This  is  a  matter  of  con¬ 
cern  since,  even  in  periods  of  shipboard  deployment,  there 
may  be  periods  of  a  week  or  more  between  landings. 

There  now  is  an  active  program  to  develop  low  cost^ 
simulation  equipment  which  can  be  used  to  maintain  carrier 
landing  proficiency,  once  it  has  been  achieved.  The 
objective  is  to  use  the  advanced  mini-computer  and  micro¬ 
processor  technologies  of  today  to  produce  small  trainers 
which  are  easily  transported  and  can  be  moved  with  a 
squadron  and  used  in  a  ready  room  environment.  These 
trainers,  in  which  no  motion  would  be  provided,  will  pre¬ 
sent  the  necessary  visual  information,  respond  to  pilot 
control  movements,  and  provide  automatic  scoring  of 
performance.  The  real  attractiveness,  of  cburse,  is  one  of 
economy,  as  shown  in  Exhibit  17.  These  trainers,  which  we 
believe  will  do  an  excellent  job  in  maintaining  carrier 
landing  proficiency,  can  be  acquired  at  a  cost  roughly 
one  one-hundredth  that  of  the  more  sophisticated  simulators. 


Conclusions 

The  goal  of  the  Navy  Training  and  Personnel  Technology 
Program  is  to  provide  the  most  effective  personnel  support 
for  the  operation  and  maintenance  of  Navy  systems.  The 
purpose  of  this  review  is  to  describe  the  program  and  its 
objectives .  Most  important,  this  review  emphasizes  that  the 
simple  acquisition  of  new  weapon  systems,  neW  training 
devices,  or  additional  manpower  does  not  in  itself  improve 
military  readiness.  Major  improvements  in  our  hardware  and 
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manpower  resources  require  corresponding  advances  in 
personnel  technology.  To  achieve  a  reduction  in  flight  hour 
costs,  to  improve  the  retention  of  airdrewmen,  to  increase 
the  effectiveness  of  new  Weapons  -  all  involve  an  increased 
commitment  by  the  personnel  arid  training  research  communi¬ 
ties  and  by  those  responsible  for  program  budgeting.  We 
feel  we  are  meeting  this  commitment  through  the  development 
of  a  program  which  is  using  aiidvariced  technology  and  which 
is  striving  for  application  at  an  affordable  cost. 


(Dollars  in  Millions) 
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Fundi  lupwiint  portion  of  62757N:  Triinino  and  Human  Enginaaring  Tachnology  allocatad  for  flight  simulator 
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Personnel  and  Training  T echnology 


if  Human  factors  ▲  Education  and  training 


'k  Human  factors  ▲  Education  and  traim^^^^ 
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Exhibit  7 


•  Pay-off  demonstrations 

improved  performance 
Decreased  costs 
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#  Increased  Tri-service  cooperation 


Fleet  Involvement 


#  Fleet  Project  Team 

•  Fleet  Implementation  Team 


Adapted  from  Qrlansky  and  String,  1977 


Aviation  Wide  Angle  Visual  System 


A-7  Fleet  Landing  Performance  Scores 
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Nugget,  No  NCLT 


Low  Cost  Niaht  Carrier  landing  Trainer 


Cost  =  3.5  M 
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LCCIM;  A  Model  for  Analyzift^g  the  Impact  of  Design 
On  ffeapon  System  Support  Requ i reflient s  and  LCC 

Abstract 


This  paper  describes  a  life  cycle  cost  impact  model¬ 
ing  system  (LCCIM)  which  resulted  from  an  effort  to  assess 
the  potential  impact  of  the  Digital  Avionics  information 
System  (DAIS)  concept  of  avionics  integration  on  weapon 
system  life  cycle  cost  and  system  support  personnel 
requirements.  Applicable  to  both  new  and  operational 
systems,  it  allows  life  cycle  cost  and  human  resources 
requirements  to  be  used  more  effectively  as  guideline 
considerations  within  both  system  design  and  modification 
processes. 

The  LCCIM  consists  of  three  models  and  associated 
data  banks  which  Operate  either  interactively  or  indepen¬ 
dently,  The  first,  a  Reliability  and  Maintainability 
(R&M)  model,  is  an  average  vaiue  mbdel  which  traces  and 
accounts  for  support  maintenance  Operations  at  either  the 
line  replaceable  unit  (LRU;)  ,  subsystem,  or  system  level  to 
produce  point  estimates  of  human  resources  requirements. 

It  can  also  identify  sources  of  high  resouree  Consumption 
and  answer  "what  if"  quest ions  concerning  the  results  to 
be  expected  from  changing  the  values  of  R&M  parameters. 

The  second  model,  a  graining  Reguirements  Analysis  model, 
examines  user  defined  tasks  and  eyaluates  them  on  the 
basis  of  five  task  ^characteristic  parameters.  It  yields 
detailed  guidance  in  the  establis'hment  of  system  support 
personnel  training  requirements.  IRased  on  the  user's 
selection  of  data  inputs  and  choice  of  optional  decision 
rules,  the  model  identifies  tasks  vrequiring  training, 
generates  a  suitabie  training  plan,  and  specifies  a 
feasible  training  program.  The  third  model  is  a  System 
Cost  model  which  aggreqates  the  epmponents  of  system  life 
cycle  cost  and  presents  them  either  in  selective  combina¬ 
tion  or  summary  form. 


BACKGROUND  ^  TV 

Confronted  with  a  reduced  t)udget,  increased  opera¬ 
tions  and  support  (O&S)  maintenance  costs,  and  a 
volunteer  force,  the  Air  Force  is  rebbgni zing  the  need  to 
reflect  these  considerations  in  the  systems  acquisition 
process.  In  the  past,  the  acquisitioh  of  a  major  system 
has  taken  from  10  to  17  years.  The  process  has  been 
basically  "open-loop"  in  the  sense  that  actual  assess¬ 
ment  of  O&S  costs  only  occurred  after  the  system  was 
deployed.  Even  if  design  or  support  improvements  could 
be  found  then,  the  retrofit  bill  was  often  prohibitive. 

This  research  was  undertaken  in  response  to  the  need 
to  discipline  the  systems  acquisition  process,  particularly 
for  avionics  systems.  The  fundeunental  objective  was  to 
develop  a  method  for  "closing  the  loop"  to  influence 
avionics  system  design  based  upon  downstream  life  cycle 
cost  and  system  support  perspnnel  requirements.  A  litera¬ 
ture  search  was  conducted  to  determine  the  availability  of 
suitable  models,  it  confirmed  that  hundreds  of  LCC  models 
do,  indeed,  exist.  However/  almost  without  exception,  they 
apply  cost  factors  to  the  expected  values  of  the  system 
variables  and  aggregate  the  cost  elements  to  determine 
total  LCC. 

In  addition  to  their  non-analytical  nature,  the  in¬ 
effectiveness  of  these  models  in  actually  reducing  system 
LCC  can  be  traced  to  the  fact  that  they  were  often  applied 
in  evaluations  which  can  only  be  described  as  post  hoc. 

These  evaluations  assessed  system  LCC  after  the  fact, 
lending  little  insight  to  means  for  cost  avoidance.  In 
many  instances,  life  cycle  cost  studies  merely  amounted  to 
cost  accounting.  In  all  fairii®ss,  it  must  be  mentioned  that 
design-to-cost  concepts  were  sometimes  employed  in  an  effort 
to  reduce  acquisition  costs.  Unfortunately,  without  consi¬ 
deration  of  logistics  support  costs,  design -to -Cost  as  it 
was  previously  applied,  often  opposed  the  life  cycle  cost 
goals  it  was  intended  to  promote.  Furtheirmore,  there  had 
been  no  comprehensive  planned  approach  to  develop  and 
apply  Cost  avoidance  ideas  early  in  the  systems  acquisition 
process,  when  such  information  could  be  acted  upon  to  result 
in  real  benefits. 

In  brief,  a  need  was  identified  for;  (1)  a  modeling 
system  and  associated  data  banks  capable  of  determining 
system  LCC  analytically,  on  an  a  priori  basis;  and  (2)  a 
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method  for  "closing  the  loop"  tp  apply  life  cycle  cost 
and  syst^  support  personnel  pequirem^ts  considerations 
at  each  phase  of  the  systems  aqquisitt ion  process.  The 
fundamental  goal  of  this  paper  is  to  describe  the  LCCIMj 
a  means  for  inf luencing  the  ^esigp  of  the  entire  integra- 
ted  system  in  terms  of  bqtfi  pperationai  effectiveness  and 
LCC .  This  viewpoint  recognizes  both  the  system  design  and 
the  total  pperatipn  and  support  system. 

approach 

A  systems  spprpapih  was  taken  to  develop  the  LCCIM. 

By  systems  approach,  we  msan  thst  the  f pi lowing  basic 
steps  were  t§kep:  ([l|  t|l:P  P?Ph|p^  objective  was  stated 
^^^-^hest  level;  (|)  interactions  between  the  major 
total  system  elements  wpfp  lienfified;  (3 )  transfer 
functions  between  various  iiiput  and  p^  variabies  were 

^i^®nts  tii^t  t|ie  designer  can  change  were 
modeled  in  terms  of  their  ip.fiuenGe  on  the  pverall  system. 

The  highest  level  problem  statement  for  the  LCCIM  is 
given  in  Figure  1.  The  iask  is  to  influence  selection 
between  acceptable  alternatiyis  fptf  (1)  system  designs, 

(2)  support  cpncepts,  and  (3)  training  and  aiding  policies 
based  upon  LCC  considerations .  fp  formulate  the  problem, 
the  overall  objective  function  was  taken  as:  Minimize  LCC 
subject  to  a  specified  effectiveness  constraint. 

_  The  Slain  Objective  Pf  Ilf®  cycle  costing  is  to  con— 
sider  system  ownershijp  as  well  as" acquisition  posts,  in 
order  to  provide  a  comprehensive  visibility  into  the 
relative  economic  advantages  of  atternative  designs.  In 
order  to  meet  this  objective,  decision  making  criteria 
pertinent  to  each  phase  pf  th®  Systems  acquisition  process 
must  be  generated  on  the  basis  pf  data  available  at  that 
point  in  time.  Clearly,  a  requirement  exists  for  more 
comprehensive  models  tp  identify  the  ownership  cost  drivers 
within  emerging  systems  and  portray  their  interactions. 

Data  in  the  hiunan  resource  areas  pf  manpower,  training, 
technical  document at ion,  and  support  equipment  need  to  be 
provided  earlier  and  in  more  detail  to  allow  for  a  timely 
and  accurate  estimation  pf  pwnership  cpsts .  Most  important- 
model  (s)  must  bp  capable  pf  operating  on  the  data 
tnat  IS  available  early  in  the  system  acquisition  process 
to  provide  cost  estimates  at  thet  particularly  critical 
time.  The  hierarchy  pf  life  cycle  costs  utilized  for  the 
development  of  bhe  LCCIM  is  shown  in  Figure  2. 


The  major  system  elements  which  drive  the  LCCIM  cost 
hierarchy  were  taken  to  bej  (4.)  the  avionics  system 
design  characteristics,  ‘t2)  the  operating  and  support 
requirements  dictated  by  the  particular  design  configure »- 
tion,  and  (3)  the  training  and  aiding  policy  necessary  to 
provide  the  required  personnel  skills  and  knowledge  for 
operation  and  maintenance.  Art  overview  of  the  procedure 
utilized  to  quantify  the  interrelationship  between  these 
major  system  elements  is  given  in  Figure  3.  The  five 
steps  shown  there  are;  CD  functional,  (2)  maintenance, 

(3)  reliability  and  maintainability  (R&M),  C4J  training, 
and  (5)  cost  analyses.  Basically,  the  equipment  drives 
the  maintenance  requirement/  the  reliability  determines 
the  demand  on  the  maintenance  System;  the  equipment 
characteristics,  R&M  parameters,  and  manpower  determine 
the  required  resources  and  associated  skills  and  knowledge 
requirements;  the  training  program,  associated  job  aids, 
support  equipment,  and  appropriate  manpower  provides  the 
required  support  capability;  and  the  cost  analysis  identi¬ 
fies  the  specific  costs  associated  with  each  element  pre¬ 
viously  shown  in  Figure  2.  Each  step  is  now  described  in 
some  detail. 

A  generic  mod^  for  avionics  suites  was  constructed 
based  upon  the  functional  requirements  for  a  representa¬ 
tive  close  air  support  (CAS)  mission.  It  was  determined 
that  the  following  functional  groups  of  equipment  were 
required;  navigation,  communications,  counter-measures, 
air-to-ground  attack,  control  and  display,  and  flight 
control.  The  process  of  its  construction  is  fully  described 
in  AFHRL-TR-76-59,  Mid-1980s  Digital  Avionics  Information 
System  Conceptual  Design  Configuration.  An"equipment 
hierarchy  was  then  established  to  describe  a  generic 
avionics  suite.  The  levels  in  the  hierarchy  consist  of 
system,  functional  group,  operational  function,  subsystem, 
and  LRU.  Following  this,  a  coding  system  was  assigned  so 
that  each  element  in  the  generic  avionics  suite  could  be 
rapidly  identified  and  indexed.  Figure  4  illustrates  the 
technique  by  showing  a  portion  of  the  equipment  hierarchy. 
For  example,  the  highest  indenture  denoting  system  level 
(avionics)  is  coded  in  the  first  space  of  the  code  designa¬ 
tion  (A).  The  functional  group  (e.g.,  communications)  is 
coded  in  the  second  space  (AC) ,  and  so  on.  Thus  the 
equipment  hierarchy  of  any  avionics  suite,  or  system,  can 
be  described  on  a  common  basis  which  allows  it  to  be 
modeled. 
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The  next  step  wes  to  model  the  operationa,!  and  main¬ 
tenance  process .  The  initial  approach  taken  w^as  to  simulate 
the  detailed  O&M  process  shown  in  Figure  5,  using  a  Monte 
Carlo  simulation  model  called  the  Logistics  Composite 
Model  (LCOM) ,  However,  due  to  a  need  for  computational 
speed  and  a  requirement  that  the  model  be  operable  on  data 
of  lesser  detail  than  that  required  for  the  LCOM,  the  R&M 
model  was  developed.  It  is  based  on  a  simplified  repre¬ 
sentation  of  the  O&M  process  as  shown  in  Figure  6.  It 
should  be  noted  that  the  operational  scenario  and  the 
maintenance  environment  are  modeled  separately.  Basically, 
the  operational  scenario  is  modeled  as  creating  a  demand 
upon  the  maintenance  system  as  a  function  of  the  niamber  of 
sorties  flown  (or  of  flying  hours)  and  the  failure  rates  of 
the  individual  equipments  in  the  avionics  suite.  The  R&M 
model  computes  the  demand  placed  on  the  maintenance  system 
on  an _LRU-basis  and  then  aggregates  to  determine  the  total 
demand.  Therefore,  the  R&M  model  treats  the  operational 
scenario  in  t^m^s  of  the  mean  flying  hours  between  main ten- 
of  individual  LRUs.  This  mean  value  of  demand 
on  the  maintenance  system  is  sufficient  for  assessing  support 
resources  during  the  conceptual  phase  of  the  acquisition 
process  and  is,  in  all  probability,  the  best  figure  which 
can  be  generated  on  the  basis  of  data  available  during  that 
time  period. 

Given  that  a  demand  is  placed  upon  the  maintenance 
system,  the  maintenance  process  must  restore  the  equipment 
to  operational  readiness.  This  is  accomplished  by  minor 
on-aircraft  repair  or  by  replacement  with  an  operationally- 
ready  LRU.  However,  since  total  support  resources  must  be 
estimated,  the  R&M  model  must  also  provide  estimates  of  the 
resources  required  for  the  repair  of  the  LRUs  in  the  shop. 

The  basic  approach  was  to  determine  all  possible  main¬ 
tenance  outcomes  or  events  that  could  result  from  a  specific 
equipment  failure.  Each  maintenance  event  places  a  demand 
on  the  maintenance  system.  The  average  resources  demanded 
by  each  maintenance  event  are  determined  on  ah  LRU-basis, 
e.g.,  maintenance  crew  composition,  support  equipment,  and 
time.  Finally,  the  probability  of  each  specific  maintenance 
event  occurring  (per  sortie  or  per  1000  flying  hours)  is 
introduced.  Total  support  resources  per  LRU  are  determined 
by  multiplying  appropriate  probabilities  by  the  support 
resources  associated  with  each  maintenance  event.  Required 
support  resources  are  then  computed  by  LRU,  Subsystem, 
functional  group,  and  total  system  by  summing  across  the 
appropriate  levels  in  the  equipment  hierarchy. 
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Later  in  the  systems  acquisition  process  when  more 

detailed  data  concerning  sysliem  design  and  operational^ 

utilization  become  availa,ble/ :  the  LCCIM  user  may 

substitute  the  previously  mentioned  LCOM  in  place  of  the 

R&M  model  within  the  LCCIM.  This  will  enable  him  to^;  ; 
examine  more  closely  the  critical  drivers  of  post  and  ^ 

operational  availability  identified  by  the  R&M  model. ^  It 
will  also  allow  him  to  account  for  peak  loads,  saturations, 
queues,  or  other  nonlinear  constraints  that  exist  in  tne 
actual  maintenance  environment  but,  are  not  picked  up  by 
the  average  value  process  of  the  R&M  model. 

The  R&M  model  described  above  provides  a  means  for 
computing  the  required  support  resources  for  a  particular 
system  design  or  support  concept  alternative.  The  next  _ 

step  in  the  LCCIM  procedure  to  quantify  the  interaction  of 
major  system  elements  addresses  maintenance  support  per¬ 
sonnel  training  requirements.  In  order  to  assess  the  _ 
impact  of  system  design,  operation,  and  support  on  train¬ 
ing  requirements,  a  training  requirements  analysis  model 
and  associated  data  bank  were  developed.  The  model  allows 
a  training  analyst  to  assign  values  to  variables  describing 
systems,  policy,  training  operations,  resources,  and  cost. 
Within  the  bounds  of  the  user  established  set  of  constraints, 
it  produces  an  estimate  of  the  training  program  pquirement 
which  their  interactions  generate.  Results  may  be  refined 
by  iteratively  exercising  the  model  using  different  values 
for  constraint  parameters  aud/or  input  data.  The  means  to 
relate  system/policy/resources/cost  input  data  to  resultant 
training  impacts  are  contained  in  the  model,  and  it  is 
programmed  for  both  user  interactive  operation  via  remote 
terminal  facilities  and  batch  operation. 


The  training  model.  Figure  7,  consists  of  three  modules: 
a  pre-processor,  and  two  analytical  modules  for  training 
plan  and  training  program  generation. 

Operation  of  the  model  is  predicated  upon  the  establish¬ 
ment  of  a  data  bank  containing  the  set  of  tasks  to  be 
learned.  Their  level  of  specificity  is  a  user  defined^ 
variable,  allowing  for  the  flexibility  of  task  definition. 
Each  task,  however,  is  assigned  five  descriptor  values 
denoting;  frequency,  criticality,  learning  difficulty, 
taxonomy,  and  sequencing.  . 

The  data  bank  is  inputted  to  the  pre-processor  module 
which  screens  the  total  set  ,pf  tasks,  in  a  series  of  go 
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no-go  decisions,  to  select  those  which  require  training. 

The  selected  tasks  then  beCdirte  the  subset  of  tasks  that 
are  the  training  requirement.  The  selection  is  based  upon 
pre-established  descriptor  value  levels  detemined  by  the 
user.  For  example,  a  criteria  Of  tasks  of  a  difficulty 
level  above  .60  may  be  used  tO  discriminate  between  tasks 
bn  the  basis  of  that  parameter.  Thus,  the  user  maintains 
control  of  the  decision  process  by  his  selection  of 
decision  criteria,  i.e.;  parameter  combinations  and  para¬ 
meter  value  cut-off  points.  The  list  of  tasks  which  the 
pre-processor  determines  to  be  requirements  for  training 
retains  its  associated  set  of  descriptor  values  and  be¬ 
comes  the  input  data  set  for  the  first  analytical  module 
which  is  the  training  plan  generator. 

At  this  point,  it  is  assumed  that  all  of  the  out¬ 
putted  tasks  are  to  be  trained.  The  user  now  has  the 
option  of  designating  a  Value  for  any  one  of  three  con¬ 
straining  Conditions:  personnel  required  (number) j  maximum 
allowable  training  cost  (dollars);  or  maximum  allowable 
training  time  (months).  He  need*  howeVer,  only  Specify 
the  training  personnel  requirement  to  operate  the  module 
using  interhalized  data  and  relationships.  The  training 
plan  generator  then  produces  an  initial  training  plan. 

This  is  a  two  Step  process  in  which  a  minimum  cOst  school/ 
On-the-job  training  (OJT)  mix  is  determined,  followed  by 
recommendations  concerning  appropriate  methods  and  media, 
e.g.,  lecture,  simulation,  mockups,  actual  equipment,  etc. 

After  reviewing  the  initial  training  plan,  the  user 
may  either  select  a  different  set  of  decision  criteria  and 
exercise  the  training  plan  generator  module  to  obtain 
another  training  plan,  or  continue  on  to  the  second  analy¬ 
tical  module  to  generate  a  training  program.  Generally, 
the  training  plan  generator  will  be  iterated  several  times 
by  the  user  as  an  investigative/Optimization  procedure 
prior  to  the  selection  of  a  training  plan  to  be  examined 
in  more  detail.  To  facilitate  this  kind  of  activity,  all 
modules  of  the  training  requirements  model  are  programmed 
for  user  interactive  operation  via  remote  terminal  faci¬ 
lities,  as  well  as  for  batch  operation. 

Having  provided  for  the  computation  of  support  resource 
requirements  and  training  program  definition,  the  last  step 
in  the  LCCIM  procedure  is  the  assignment  of  appropriate 
cost  factors  for  each  variable.  Referring  again  to  Figure 
2,  each  cost  element  in  the  hierarchy  must  be  quantified 
and  aggregated  to  evaluate  the  total  cost  impact  of 


alternative  system  designis  anS  support  concepts  under 
evaluation.  This  is  accomplished  by  means  of  a  system 
cost  model  component  of  the  bCGIM  vhich  is  both  user 
interactive  and  operable  interactively  with  the  R&M  model 
via  remote  terminal  facilities^  and  operable  either  singly 
or  interactively  with  the  R&M  model  in  a  batch  processing 
mode  of  operation. 

The  systems  approach  employed  by  the  LCCIM  consists 
of  a  structured  process  that  provides  for  the  efficient 
use  of  available  information.  That  process  recognizes  the 
incompleteness  and  inexactitude  of  the  data  existing  during 
the  conceptual  phase  of  the  systems  acquisition  process 
that  must I  however »  be  used  to  forecast  outyear  resource 
utilization  and  cost.  Within  it  a  statement  of  the  basic 
need  for  the  weapon  system  leads  to  the  idetttif ication  of 
the  most  comparable  reference  system/  modification  of  ref¬ 
erence  data  to  reflect  technplogical  advances  and  advanced 
O&S  concepts  produces  baseline  input  data  for  the  LCCIM/ 
and  those  data  are  then  processed  to  determine  resource 
utilization  in  terms  of  man  and  machine  requirements. 

SUMMARY 

The  LCCIM  provides  a  powerful  analytical  tool  which  is 
particularly  suited  to  an  investigative  role  in  determining 
how  to  guide  the  design  and  support  of  systems  to  achieve 
essential  capabilities  at  affordable  cost.  This  is  true 
throughout  a  system's  life  cycle*  from  conception  and 
including  outyear  modification.  As  greater  system  defini¬ 
tion  data  become  available  the  LCCIM  can  transition  from 
its  basic  impact  mode  of  operation  to  one  which  enables  the 
detailed  analysis  of  system  cost  and  requirements. 

An  overview  of  LCCIM  operation  is  given  in  Figure  8. 
The  total  system  combines  the  R&M/  training  requirements 
analysis^  and  system  cost  models  to  assess  the  LCC  impacts 
of  various  design*  support,  and  training  alternatives. 
Operable  in  either  an  interactive  or  batch  processing  mode, 
the  ability  of  the  LCCIM  to  be  effectively  used  iteratively 
by  the  system  designer  allows  for  systematic  assessments 

objectives  and  trade—off  studies  of  alternative 
designs.  The  model  outputs  are  presented  in  formats  which 
provide  general  (top  down)  and  detailed  (bottom  up)  per¬ 
spectives,  as  well  as  visibility  at  the  intermediate  levels 
of  system  cost  and  resource  impact  assessment. 
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Outputs  are  developed  by  aggregating  resource  utiliza¬ 
tion,  applying  cost  factors,  and  grouping/ranking  measures 
of  impact  CMOI)  such,  as  maintenance  manhours  per  flighthohr 
or  flightline  service  availability.  High  resource  impacts 
located  at  the  top  Of  the  MOI  lists  can  be  identified  as 
areas  in  which  changes  can  produce  significant  payoffs  in 
cost  avoidance.  Output  data  can  be  examined  at  various 
levels  of  detail  to  identify  dominant  resource  and  cost 
drivers.  Sensitivity  analyses  can  be  conducted  within  the 
modeling  system  to  measure  the  effect  of  interrelationships 
among  model  parameters. 

After  identifying  dominant  drivers  and  determining 
parameter  sensitivities,  the  LCCIM  can  be  used  to  choose 
between  "finalist"  candidate  desigh  alternatives  which 
are  "tuned  up"  versions  of  alternatives  evaluated  pre¬ 
viously  by  the  LCCIM.  These  Could  represent  either  single 
subsystems,  modified  oh  the  basis  of  information  provided 
by  the  LCCIM,  or  a  composite  of  such  subsystems,  ih  this 
final  step,  all  parameters  affected  by  the  revision  of 
alternatives  would  be  changed  accordingly  and  the  LcClM 
would  be  Operated  as  a  unit  to  provide  a  final  total  LCC 
comparison.  Thus,  the  trade-‘Off  process  can  be  followed 
to  completion  in  comparing  major  system  alternatives  as 
well  as  in  making  a  series  of  gradual  parameter  chahges 
that  lead  to  a  set  of  design  or  support  planning  characf 
teristics  that  best  satisfies  the  basic  need  at  an  afford »- 
able  cost. 
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Figure  5  OPERATIONS  AND  MAINTENANCE  PROCESS 
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Figure  6  R&M  OPERATIONS  AND  MAINTENANCE  PROCESS 
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Figure  8  OVERVIEW  OF  LGCIW  OPERATION 
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PACTS;  Use  of  Individualized  Automated 
Training  Technology 

Abstract 

The  cost  effectiveness  of  Individualized  Automated 
Training  (lAT)  is  readily  seen  in  reduced  instructor  require- 
ments,  increased  training  standardization,  increased  trainee 
prof iciency ,  and  increased  trainee  throughput.  The  PACTS 
program  which  combines  computer  speech  recognition,  computer 
speech  synthesis,  automated  performance  measurement,  auto¬ 
mated  syllabus  control,  and  instructor  modeling  is  one 
example  of  successful  lAT  in  the  laboratory.  PACTS  techno¬ 
logy  has  advanced  to  6.4  for  field  evaluation  at  the  Navy's 
Air  Traffic  Control  School  and  the  Fleet  Combat  Trainincj 
Center,  Pacific. 

PACTS  provides  automated  subsystems  to  introduce  the 
trainee  to  his  task  by  combining  voice  technology  with  hands- 
on  practice.  Following  a  run,  voice  technology  is  combined 
with  models  of  the  instructor  to  provide  automated  critique 
of  trainee  performance.  Thus,  PACTS  frees  the  instructor  of 
routine  tasks,  while  at  the  same  time  increasing  traininq 
standardization. 
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Introduction 

Individualized  automated  training  has  a  number  Of  advan¬ 
tages  over  more  traditional  approaches  to  training.  Automa¬ 
tion  of  training  relieves  the  instructor  of  busywork  chores 
such  as  equipment  setup  and  bookkeeping.  He  is  thus  free  to 
use  his  time  counseling  students  in  his  role  asi  training 
manager.  Individualized  instruction,  with  its  self-paced 
nature  maintains  the  motivation  of  the  trainee.  Objective 
scoring  is  potentially  more  consistent  than  subjective  ratings. 
Uniformity  can  be  maintained  in  the  proficiency  level  of  the 
end  product,  the  trainee.  But,  tasks  requiring  verbal  com¬ 
mands  have  thus  far  been  unamenable  to  individualized  auto¬ 
mated  training  techniques.  Traditionally,  performance  measure¬ 
ment  of  verbal  commands  has  required  subjective  ratings.  This 
has  effectively  eliminated  the  potential  development  of  indi^ 
vidualized,  automated  self-paced  curricula  for  the  training 
of  the  Landing  Signal  Officer  (LSO) ,  the  Air  Intercept  Control¬ 
ler  (AIC) ,  the  Precision  Approach  Radar  (PAR)  Controller,  and 
others.  Computer  recognition  of  human  speech  offers  an  alter¬ 
native  to  subjective  performance  measurement  by  providing  a 
basis  of  objectively  evaluating  verbal  commands.  The  current 
state  of  the  art  has  allowed  such  applications  as  automated 
baggage  handling  at  Chicago's  O' Hare  airport,  A  more  sophis¬ 
ticated  recognition  System  is  required  for  training,  however. 

To  that  end,  the  Naval  Training  Equipment  Center,  Human  Factors 
Laboratory  Prototype  Automated  Controller  Training  System 
(PACTS)  program  will  provide  two  separate  experimental  systems, 
one  for  PAR  and  one  for  AIC. 

PACTS  Training  Requirements 

The  PAR  Application 

The  task  of  the  PAR  controller  is  to  issue  advisories  to 
aircraft  on  the  basis  of  information  from  a  radar  indicator  con 
taining  both  azimuth  (course)  and  elevation  (glidepath)  capa¬ 
bilities.  The  aircraft  target  projected  on  the  elevation  por¬ 
tion  of  the  indicator  is  mentally  divided  into  sections  by  the 
controller.  This  is  because  the  radio  terminology  (R/T)  for 
glidepath  is  defined  in  terms  of  these  sections.  Thus,  at  any 
one  point  in  time,  one  and  only  one  advisory  is  correct.  Con¬ 
versely,  each  advisory  means  one  thing  and  only  one  thing. 

This  tightly  defined  R/T  is  perfect  for  application  of  objec¬ 
tive  performance  measurement.  The  drawback,  of  course,  is  that 
performance  is  verbal  and  has  thus  far  required  subjective 
ratings.  In  addition,  the  time  required  for  hxaman  judgement 
results  in  inefficient  performance  measurement.  The  instructor 
cannot  catch  all  the  mistakes  when  there  are  many  of  them. 
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The  major  behavioral  objective  of  current  PAR  training 
is  to  develop  the  skill  to  observe  the  trend  of  a  target  and 
correctly  anticipate  the  corrections  needed  to  provide  a  safe 
approach.  The  standard  R/T  is  designed  to  provide  a  medium 
to  carry  out  this  objective,  and  PAR  training  exposes  the 
student  to  as  many  approaches  as  possible  so  that  the  trainee 
may  develop  a  high  level  of  fluency  with  his  R/T. 

The  primary  need  to  fulfill  its  objective  is  for  PAR 
training  to  teach  the  skill  of  extrapolation.  A  controller 
must  recognize  as  quickly  as  possible  what  the  pilot's  skill 
is.  He  must  recognize  what  the  wind  is  doing  to  the  aircraft 
heading.  Then  he  must  integrate  this  with  the  type  aircraft 
to  determine  what  advisories  to  issue. 

The  AIC  Application 

The  task  of  the  AIC  is  somewhat  more  complex  than  that 
of  the  PAR  controller.  In  fact,  the  AIC  has  a  job  of  multi¬ 
ple  tasks  which  are  complex  in  terms  of  concepts  and  decision 
processes,  as  well  as  response  sequences.  There  are  multiple 
aircraft  to  be  monitored,  and  some  of  them  may  not  be  friend¬ 
ly.  Complicated  geometry  must  be  used  in  the  set-up  of  an 
intercept  so  that  the  enemy  will  not  get  the  tactical  advan¬ 
tage.  A  multitude  of  information  such  as  heading,  speed, 
altitude,  and  fuel  state,  as  well  as  much  more,  can  be  mon¬ 
itored  if  the  AIC  pushes  the  correct  button  on  the  computer¬ 
ized  console. 

There  are  three  basic  behavioral  objectives  of  current 
AIC  training,  and  numerous  supporting  objectives.  A  trainee 
must  be  capable  of  learning  to  locate  a  specific  aircraft  in 
the  midst  of  ground  clutter,  clouds,  and  other  aircraft  sym¬ 
bols,  build  a  symbol  for  that  aircraft,  then  perform  the 
appropriate  actions  to  cause  the  engagement  of  the  aircraft 
from  the  computerized  radar  console.  After  that,  the  AIC 
must  verify  the  position  updating  of  the  computer  controlled 
symbol  for  all  the  targets  on  the  console  and  establish  a 
rhythm  for  all  AIC  actions.  A  trainee  must  also  be  capable 
of  learning  to  recognize  changes  in  the  enemy  target  para¬ 
meters  such  as  altitude,  airspeed  or  heading,  then  notify  the 
pilot  of  the  interceptors.  In  addition,  unknown  aircraft  may 
enter  the  area  in  such  a  way  as  to  create  a  potential  conflict 
with  the  intercept,  and  the  pilot  must  be  kept informed  of  the 
actions  of  these.  Finally,  a  trainee  must  be  capable  of  learn¬ 
ing  to  provide  changes  in  heading  to  aircraft  which  are  practic 
ing.  Often,  the  AIC  is  called  upon  to  monitor  pilots  who  want 
to  practice  intercepts,  taking  turns  being  the  enemy  for  one 
another.  In  this  situation,  the  AIC  must  contain  the  aircraft 
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within  the  geographic  practice  area  as  well  as  issue  head¬ 
ing  changes  to  set  up  the  intercept  for  both  aircraft. 

Advanced  technology 

The  major  behavioral  objectives,  then,  can  more  effi¬ 
ciently  be  achieved  through  the  application  of  computer 
speech  recognition  technology,  and  thereby  the  application 
of  advanced  training  technologies.  This  is  because  with 
objective  assessment  of  what  the  controller  is  saying,  objec¬ 
tive  performance  measurement  is  possible,  and  thus  we  have 
the  capability  of  individualized  instruction.  The  use  of 
simulated  envirorunental  conditions  allows  the  development  of 
a  syllabus  of  graduated  conceptual  complexity.  The  integra¬ 
tion  of  these  components  results  in  an  automated  self-paced, 
individualized  training  system. 

The  job  of  the  instructor  now  becomes  one  of  training 
manager.  His  experience  and  skill  may  be  exploited  to  its 
fullest.  The  training  system  can  provide  support  in  intro¬ 
ducing  the  student  to  the  R/T.  The  instructor  can  scan  the 
progress  of  each  student,  then  provide  counseling  to  those 
who  need  it.  Routine  error  feedback  is  provided  by  the 
training  system.  Only  the  instructor  can  provide  human  to 
human  counseling  for  specific  needs,  and  the  training  system 
provides  more  time  for  this  valuable  counseling. 

Training  System 

Three  major  constraints  are  imposed  by  this  system. 

Each  user  must  pre-train  the  phrases.  Recognition  does  not 
take  place  for  random,  individual  words,  only  for  predefined 
phrases.  Each  phrase  is  repeated  a  number  of  times  and  a 
Reference  Array  is  formed  representing  the  "average"  way  this 
speaker  voices  this  particular  phrase.  Thus,  the  second  con¬ 
straint  is  that  there  must  be  a  small  number  of  phrases 
(about  50)  which  are  to  be  recognized.  If  performance  is  to 
be  evaluated  based  upon  proper  R/T,  each  phrase  must  be  de¬ 
fined.  The  third  constraint,  due  to  performance  measurement 
requirements,  is  that  there  be  no  ambiguous  phrases  —  right 
or  wrong  depending  strictly  on  who  the  instructor  is.  Tech¬ 
nically,  the  PACTS  application  appears  to  be  conformable  to 
these  constraints. 

To  achieve  high  fidelity,  simulation  makes  use  of  var¬ 
ious  models:  The  model  of  the  controller  is  at  the  focal 
point  of  all  other  models,  and  serves  to  provide  criteria  to 
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the  performance  measurement  system.  A  model  of  the  aircraft 
and  pilot  allows  for  variation  in  the  complexity  of  situa¬ 
tions  presented  the  student.  The  principle  being  used  here 
is  that  exposure  of  the  student  to  certain  typical  situations 
will  allow  him  to  generalize  this  experience  to  real  world 
situations.  The  pilot  model  allows  for  systematic  presenta¬ 
tion  of  various  skill  levels  of  pilots.  In  addition,  the 
equations  used  in  modeling  the  pilot  and  aircraft  responses 
also  allow  for  introduction  of  various  wind  components.  The 
adaptive  variables  of  pilot  skill,  aircraft  characteristics, 
and  wind  components  are  combined  systematically  to  produce 
a  syllabus  graduated  in  problem  complexity.  As  the  skill  of 
the  trainee  increases,  he  is  allowed  to  attempt  more  complex 
problems. 

since  the  score  is  determined  by  the  performance  measure¬ 
ment  system,  the  heart  of  scoring  is  the  model  controller. 

As  it  often  happens,  what  constitutes  "the"  model  controller 
is  a  matter  of  some  discussion  among  instructors.  Thus  for 
automated  training  applications ,  one  must  determine  the  con¬ 
cepts  which  are  definable,  such  as  how  to  compute  a  turn,  and 
leave  other  concepts  to  be  developed  by  the  instructor-student 
apprentice  relationship. 

Benefits 

Results  of  laboratory  efforts  indicate  that  training  can 
be  enhanced  and  manpower  costs  reduced  by  a  careful  integra¬ 
tion  of  advanced  training  technology  with  off-the-shelf  com¬ 
puter  speech  recognition  hardware  which  is  enhanced  with 
software  algorithms  designed  for  a  specific  vocabulary  set.. 

The  advantage  brought  to  training  by  this  technology 
is  the  capability  to  objectively  measure  speech  behavior. 
Traditional  training  techniques  for  jobs  which  are  primarily 
speech  in  nature  require  someone  who  can  listen  to  what  is 
being  said.  Otherwise,  no  direct  measure  of  the  speech 
behavior  is  possible.  In  addition  to  the  requirement  of 
having  an  instructor,  listen  to  the  speech  behavior,  train¬ 
ing  often  requires  another  person  to  cause  changes  in  the 
environment  which  corresponds  to  the  trainee's  commands.  For 
the  PACTS  tasks,  this  takes  the  form  of  "pseudo"  pilots  who 
"fly"  a  simulated  aircraft  target.  This  2:1  ratio  of  support 
personnel  to  trainee  results  in  a  relatively  high  training 
cost. 

Previous  studies  have  demonstrated  that  in  analogous 
situations,  it  has  been  possible  to  achieve  savings  of  man¬ 
power  and  training  time  while  gaining  a  uniform^  high-quality 
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student  output  by  introducing  individualized  automated  in¬ 
struction,  This  advanced  technology  would  bring  in  its 
standard  benefits  such  as  objective  performance  measurement 
and  complete  individualized  instruction.  Moreover  a  fully 
automated  system  could  provide  greater  realism  in  the  per¬ 
formance  of  "aircraft"  under  control  of  students  by  access¬ 
ing  directly  the  computer  model  of  aircraft  dynamics  rather 
than  relying  oh  the  undetermined  skills  of  a  variety  of 
pseudo-pilots.  Additionally,  the  rapid  processing  of  an 
automated  system  would  make  possible  extrinsic  feedback  of 
task  performance  to  the  trainee  in  real-time. 
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Increasing  the  Affordability  of  j-Level  Maintenance  Training 
Through  Application  of  Simulation 

Abstract 

Current  concepts  for  Air  Force  Technical  Training  in  the  maintenance 
of  today's  sophisticated  weapon  systems  rely  heavily  on  the  use  of  operational 
equipment.  This  is  especially  so  for  training  in  use  and  repair  of  the 
Automatic  Test  Equipment  (ATE)  associated  with  the  Intermediate  Level 
(I-Level)  repair  of  aircraft  avionics  systems  and  their  "black- box" 
components.  A  number  of  problems  associated  with  the  use  of  ATE 
for  training,  including  (1)  cost,  (2)  complexity,  (3)  low  reliability  when 
used  in  the  classroom,  (4)  inah^lty  to  demonstrate  realistic  malfunctions, 
have  aroused  the  Air  Force  intd  seeking  alternatives  to  the  use  of  opera¬ 
tional  equipment  for  training. 

In  the  case  of  ATE,  the  primary  alternative  currently  being  con¬ 
sidered  is  computer-based  simulation  of  the  operational  equipment.  The 
Air  Force  Human  Resources  Laboratory,  Technical  Training  Division, 
under  contract  with  the  Honeywell  Corporation,  began  in  August  1976  to 
develop  a  simulator  for  one  item  of  ATE  from  the  F-lliD  weapon  system. 

The  6883  Test  Station,  known  as  the  F-lllD  Converter /Flight  Controls 
Test  Station,  was  selected  for  this  simulation  research  effort  based  on  a 
requirement  frorri  the  Air  Force's  Air  Training  Command  (ATC)  which 
was  concerned  with  providing  adequate  "hands-on"  training  for  I-Level 
repair  activities. 

The  6883  Simulator  System  design  which  was  based  on  a  detailed 
system  specification  developed  earlier,  is  a  multi -computer  system 
which  drives  simulations  of  the  6883  Test  Station  and  associated  F-lllD 
avionics  components,  using  appropriate  hardware  interfaces.  Student 
actions  on  the  simulated  equipment  are  sensed  by  the  computer  through 
the  same  interfaces.  Guidance  and  feedback  are  provided  to  students 
via  a  CRT/keyboard  and  random  access  slide  projector.  Student 
performance  will  be  recorded  by  the  computer  system  and  output  to 
the  instructor's  CRT/keyboard  in  a  summarized  format. 

Simulation  features  incorporated  into  the  6883  Simulator  System 
include  metal  photos,  both  with  and  without  fxmctional controls /displays, 
and  simulated  printed  circuit  (PC)  cards,  using  epoxy  molding  and  color 
photography. 
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Research  questions  which  will  be  addressed  in  this  program  are 
many  and  varied.  Previous  research  in  the  area  of  maintenance  simula¬ 
tion  has  been  limited  in  scope  and  has  provided  very  few  conclusive 
answers  to  many  nagging  questions.  Although  many  studies  (reviewed 
in  the  paper)  have  emphasized  the  potential  of  simulation  in  maintenance 
training,  the  fact  remains  that  no  satisfactory  methodology  for  incorpor¬ 
ating  simulation  into  maintenance  training  systems  acquisition  has  been 
developed.  Furthermore,  there  are  a  host  of  unresolved  issues  which 
must  be  addressed  and  for  which  guidelines  must  be  developed  prior  to 
optimization  of  sim\alator  use  for  Air  Force  technical  training.  This 
paper  describes  the  variuos  issues  and  AFHRL's  approach  to  resolving 
these  issues  using  the  6883  simulator  as  a  test  bed. 


Introduction 

Teaching  students  how  to  operate  and  maintain  sophisticated 
electronic  equipment  has  always  been  a  problem  in  Air  Force  technical 
training.  Over  the  years,  various  approaches  have  been  used  includ¬ 
ing  mock-ups,  cutaways,  and  bench  mounted  actual  aircraft  parts. 
Although  varying  levels  of  success  have  been  achieved  with  the  above 
approaches,  problems  such  as  high  cost,  lack  of  reliability,  inability 
to  insert  controlled  malfunctions,  and  an  absence  of  instructional 
features  have  all  contributed  to  the  demand  for  more  c os t-and -training 
effective  approaches. 

With  the  advent  of  mini  and  micro  computer  technology  and  soft¬ 
ware  modularization,  it  has  now  become  possible  to  simulate  the  man/ 
machine  interactions  of  even  the  most  sophisticated  maintenance  equip¬ 
ment  at  a  fraction  of  the  cost  of  the  actual  equipment.  In  addition,  the 
decreasing  costs  of  peripherals  have  made  it  possible  to  incorporate 
instructional  features  into  maintenance  training  simulators  that  tend  to 
dwarf  the  instructional  capabilities  of  the  "real"  equipment. 

Although  the  age  of  simulation  technology  has  arrived,  the  know¬ 
how  required  to  effectively  utilize  these  new  developments  has  been 
lagging.  A  research  program  is  now  underway  at  the  Technical  Training 
Division  of  the  Air  Force  Human  Resources  Laboratory  to  provide  the 
missing  "know-how.  "  This  project,  which  is  called  Project  2361, 
"Simulation  for  Maintenance  Training,  "  was  designed  to  develop  simula¬ 
tion  technology  for  maintenance  training  by  providing  comprehensive 
demonstrations  of  simulation  technology  in  a  variety  of  applications. 
Lessons  learned  will  be  incorporated  into  detailed  "how  to"  guides  and 
specifications. 

The  main  thrust  of  this  paper  is  to  discuss  the  recent  develop¬ 
ment  of  a  maintenance  training  simulator  that  will  be  used  by  AFHRL 
as  a  research  test  bed  for  resolving  some  of  the  major  issues  which 
must  be  addressed  prior  to  the  optimal  use  of  simulation  in  Air  Force 
technical  training. 

Development  of  the  6883  Maintenance  Training  System 

The  primary  objectives  of  this  project  were  to  a)  design,  fabricate, 
and  test  a  simulator  for  the  6883  Converter /Flight  Controls  Test  Station; 
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b)  establish  a  research  test  bed  for  investigating  the  major  variables 
that  impact  the  design  of  training  equipment  fot  intermediate-level 
maintenance  (I-level,  or  "shop"  maintenance);  and  c)  evaluate  the 
effectiveness  of  simulation  technology  for  training  Air  Force  technicians 
for  a  wide  variety  of  checkout  and  troubleshooting  procedures  involved 
in  the  operation  and  maintenance  of  the  6883  Converter/Flight  Controls 
Test  Station  depicted  in  Figure  1. 


Insert  Figure  1  about  here 


The  6883  Converter/Flight  Controls  Test  Station  is  part  of  the  aero¬ 
space  ground  equipment  comprising  the  F-HID  avionics  I-level  maint¬ 
enance  shop.  Two  courses  related  to  I-level  maintenance  of  F- 11  ID 
avionics  are  taught  at  Lowry  Technical  Training  Center  (LTTC),  Lowry 
Air  Force  Base,  Colorado.  Course  3ABR32631D-002  trains  individuals 
to  operate  the  various  test  stations  in  the  F-lllD  shop,  including  the 
6883,  and  to  test,  inspect,  troubleshoot,  and  repair  faulty  line  replace¬ 
able  units  (LRUs)  from  the  aircraft.  Trainees  also  learn  to  perform 
limited  tests  of  the  test  station  to  ensure  that  it  is  functioning  properly. 
Course  3ABR32630B-D00/001/002  trains  individuals  to  perform  detailed 
tests;  to  inspect,  troubleshoot,  and  repair  malfunctioning  test  stations; 
and  to  perform  periodic  preventative  maintenance  on  the  test  stations. 

Current  Approach  to  I-Level  Training 

Currently,  I-level  maintenance  training  is  conducted  on  actual 
equipment.  This  approach  has  obvious  value  in  that  procurement  is 
easily  accomplished  (no  special  training  equipment  design  is  required) 
and  the  realism  provided  has  significant  motivational  value  for  both 
instructors  and  students.  However,  several  major  problems  have 
arisen  that  limit  the  training  value  of  actual  equipment  which  generally 
is  not  designed  to  be  used  in  a  training  environment.  One  primary  dis¬ 
advantage  of  using  actual  equipment  for  I-level  training  is  the  cost  of 
acquisition  and  maintenance.  Acquisition  costs  of  such  hardware  often 
exceed  several  million  dollars.  Another  disadvantage  of  using  actual 
equipment  comes  from  the  nature  of  most  I-level  jobs;  In  order  to 
train  a  student  to  become  familiar  with  a  specific  procedure,  the  actual 
equipment  must  be  run  through  highly  procedural,  time  consuming 
exercises  that  require  little  operator  participation. 
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Other  shortcomings  of  the  Ectueil  equipment  sippronch  to  treiinihg 

are; 

Extremely  low  reliability  of  the  actual  6883  test  station,  result¬ 
ing  in  low  availability  of  the  device  for  training  purposes. 

High  risk  of  severe  injury  to  trainees. 

High  risk  of  costly,  student- induced  damage  to  the  equipment. 

Limited  range  of  equipment  faults  and  emergency  conditions 
to  which  trainees  can  be  exposed,  and 

Feedback  delivery  which  necessitates  instructor's  continued 
presence. 

These  problems  limit  hands-on  procedures  training  and  troubleshoot¬ 
ing  practice.  The  6883  Maintenance  Training  System  (MTS)  was  de¬ 
signed  to  alleviate  the  above  problems  by  providing  a  much  less  costly, 
and  more  forgiving,  training  environment  that  emphasizes  job  tasks 
requiring  manual  assistance  and  troubleshooting  knowledge. 

Design  Approach 

The  design  of  the  6883  MTS  was  based  on  a  job  task  analysis  con¬ 
ducted  by  AFHRL/TT  with  the  assistance  of  ATC  instructors  who  were 
familiar  with  the  operation  and  maintenance  of  the  6883  test  station. 

This  task  analysis  was  subsequently  incorporated  into  an  AFHRL/TT 
developed  functional  specification  for  the  6883  MTS  (Miller  and  Gardner, 
1975).  A  detailed  description  of  the  task  analysis  and  methodology  for 
developing  the  functional  specification  is  contained  in  that  report. 

The  above  functional  specification  was  used  as  the  primary  contractual 
document  for  the  work  described  in  this  paper. 

The  general  approach  adopted  for  the  development  of  the  6883 
MTS  included;  a)  formation  of  a  multidisciplinary  team,  inchxding 
training  specialists,  engineers,  human  factors  psychologists,  and  sys- 
tc  m  programmers;  b)  Extensive  involvement  of  Air  Force  instructors 
and  subject  matter  experts;  c)  Refinement,  through  a  "front-end" 
analysis,  of  the  functional  specification  for  use  in  engineering  design; 
d)  incorporation  of  distributed  processing  architecture  for  expansion 
capability;  and  e)  Use  of  modular  software  to  maximize  general  application. 
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This  approach  was  organized  around  a  six- element  system  concept; 

1)  simulated  hardware,  2)  student  console,  3)  instructor  console,  4) 
computer  hardware,  5)  system  software,  and  6)  instructional  features. 
The  selected  approach  emphasized  use  of  proven,  off-the-shelf  hard¬ 
ware  and  software  elements  wherever  possible,  development  of  cost- 
effective  simulation  techniques,  and  development  of  modular  software 
to  promote  flexibility.  Figure  2  is  an  artist's  concept  of  the  simulation 
system  prior  to  fabrication. 


Insert  Figure  2  about  here 


System  Description 

This  section  provides  a  description  of  the  basic  6883  MTS  and 
a  discussion  of  the  major  design  features.  A  detailed  discussion  will 
be  provided  in  a  soOn  to  be  published  AFHRL  Technical  Report  entitled, 
"6883  Converter/Flight  Controls  Test  Station  Maintenance  Training 
System. " 

Configuration 

The  6883  MTS  is  a  dual -computer  system  which  drives  simulations 
of  the  6883  test  station  and  associated  LRUs  through  appropriate  inter¬ 
face  hardware.  The  6883  MTS,  shown  in  Figure  3,  incorporates  a 


Insert  Figure  3  about  here 


simulated  6883  test  station  (6883  simulator),  three  simulated  LRUs, 
and  simulations  of  four  associated  interface  adapters.  Student  actions 
on  the  simulated  equipment  are  sensed  by  the  computer  through  the 
same  interfaces.  Appropriate  student  guidance  and  feedback  are  pro¬ 
vided  by  a  CRT/keyboard  and  random  access  slide  projector.  Student 
performance  is  recorded  by  the  computer  system  and  is  output  to  the 
instructor's  CRT/keyboard  in  summarized  form.  These  same  perform¬ 
ance  data  can  be  output  to  a  cassette  tape  and  line  printer  for  record¬ 
keeping.  A  training  system  hardware  block  diagram  is  shown  in 
Figure  4. 
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Insert  Figure  4  about  iiere 


More  specifically,  the  6883  MTS  computer  system  architecture  is  a 
mxilti-processor,  distributed  system  providing  expansion  capabilities. 
One  Honeywell-716  computer  (H716)  functions  as  a  classroom  controller, 
operating  the  instructor  station  CRT/keyboard,  high-speed  printer, 
disk,  tape  drives,  and  interprocessor  interface. 

Data  transfer  between  the  classroom  controller  and  the  student 
station  controller  is  performed  using  an  industry  standard  RS232  inter¬ 
face.  Three  (or  more)  additional  RS2 32  interfaces  can  be  used  to 
connect  additional  test  station  simulators  to  the  classroom  controller. 
The  6883  system  computer  architecture  is  therefore  designed  to  pernlit 
expansion  to  a  total  of  four  or  more  simulated  test  stations.  In  this 
manner,  a  single  instructor  at  the  console  could  simultaneously  monitor 
several  different  station  simulators. 

The  second  H716  functions  as  the  student  station  controller.  The 
student  station  controller  operates  the  student  CRT/keyboard,  random 
access  display  unit,  interprocessor  interface,  I/O  multiplexer  and  test 
station,  and  LRU  simulated  hardware.  A  trainer  interface  electronics 
system  (TIES)  provides  an  input/output  multiplexing  capability  for 
sensing  student  actions  and  for  driving  displays  and  indicators  on  the 
simulated  equipment.  A  random-access  MAST  slide  projector  is 
computer-controlled  through  this  multiplexer. 

System  Elements 

The  6883  MTS  depicted  in  Figure  3  was  installed  at  Lowry  AFB 
in  June  1978.  Major  elements  are  the  a)  instructor  station,  b)  student 
station  and  c)  test  simulator  hardware. 

Instructor  station.  The  instructor  station  consists  of  a 
classroom  controller  and  an  instructor  console  as  shown  in  Figure  5* 

The  heart  of  the  classroom  controller  is  a  standard,  commercially 
available  H716  minicomputer  with  32,  768  words  of  internal  memory. 


Insert  Figure  5  about  here 
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other  off-the-shelf  equipment  used  within  the  classroom  controller  are 
3,  Honeywell  9030  expansion  drawer^  a  Honeywell  5400  cassette  magnetic 
tape  with  the  5401  expansion  feature  (two  cassette  tape  operation),  a 
Honeywell  4768  dual  cartridge  disk,  and  a  Honeywell  9400  power  distri¬ 
bution  unit. 

The  instructor  console  consists  of  a  Hewlett- Packa.rd  Model  (HP) 
2640B  interactive  display  terminal  (CRT),  a  Centronics  Model  102AL 
line  printer,  and  a  desk  and  chair. 

Student  station.  In  the  6883  MTS,  the  student  station, 
shown  in  Figure  6,  is  the  center  of  training  activities.  The  student 


Insert  Figure  6  about  here 


station  elements--student  station  controller,  the  student  console  (CRT/ 
keyboard  and  slide  projector),  the  simulated  6883  test  station,  and  simu¬ 
lated  6883  test  station,  and  simulated  LRUs --interact  to  provide  the 
student  with  computer-generated  responses  which  mimic  operational 
equipment  analog  (meters)  and  discrete  (lamp  and  digital  panel  meter) 
signal  responses.  Messages  displayed  on  the  interactive  CRT  termital 
supplement  simulated  operational  equipment  responses  and  guide  the 
student  through  the  correct  interpretation  of  technical  material.  Figure 
6  shows  the  student  station  controller,  interactive  display  terminal,  and 
slide  projection  system. 

Simulation  hardware.  The  6883  simulated  hardware  consists 
of: 


6883  test  station 
Three  LRUs 

Feel  and  Trim  Assembly 
Multiplexer  converter  set 
Flight  control  yaw  computer 

Four  adapters 

Three  station/LRU  interfaces 
One  station  self-test 


Insert  Figure  7  about  here 
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All  cabling  and  hoses  > 

The  6883  test  station  simulation  shown  in  Figure  7  consists  of 
28  metal  photo  panels,  three  pull-out  drdwerfe,  and  an  unmodified  GFE 
oscilloscope  mounted  in  four  salvage  GFE  equipment  racks.  These 
racks  are  mounted  on  fork  lift  support  bases,  two  racks  per  base. 
The  level  of  simulation  for  each  panel  varies.  Certain  panels  are 
complete  visual  simulation,  while  others  contain  many  functionally 
simulated  components.  Figure  8  provides  a  panel- by -panel  break¬ 
down  of  the  level  of  simulation  fidelity  oti  the  test  station.  Three  of 


Insert  Figure  8  about  here 


the  drawers  (DATAC,  power  supply,  and  flight  control  sensor)  are 
simulated  as  pull-out  drawers.  Hands-on  tasks  such  as  removal 
and  replacement  of  cards,  power  supply  adjustments,  changing  relays, 
resetting  circuit  breakers,  and  adjustment  of  potentiometers  are  fully 
simulated  in  the  three  pull-out  drawers.  Students  actually  make  the 
required  hands-on  adjustments  which  are  interactively  sensed  and 
displayed  via  the  simulated  digital  voltmeter  or  oscilloscope.  In  this 
manner,  complex  hands-on  maintenance  activities  required  of  6883 
technicians  are  simulated  with  a  high  degree  of  realism, 

LRU  simulations.  In  general,  the  exterior  of  each  of  the 
three  LRUs  is  simulated  in  appearance,  using  an  appropriate  GFE- 
salvaged  LRU  chassis  as  the  basis  for  fabrication.  These  chassis, 
stripped  of  operational  equipment,  and  the  corresponding  covers  are 
painted  to  match  the  test  station  simulator.  The  front  panels  are 
represented  using  metal  photos,  reflecting  identification  plates, 
elapsed  time  meters,  and  jack  and  switch  identifiers.  The  necessary 
functional  features  of  each  unit  are  mounted  at  the  appropriate  positions 
on  the  metal  photos.  Thefunc tional  features  of  specific  LRUs  are  detailed 
in  the  following  paragraphs.  Each  simulated  LRU  is  appropriately 
weighted  to  resemble  the  corresponding  actual  equipment.  All  simula¬ 
ted  LRUs  have  handles  in  the  appropriate  locations.  As  an  <!xamplc;, 
the  iiterior  of  the  fed  and  trim  assembly  LRU  (Figure  9)  is  simulated 
through  a  combination  of  ne  tal  photos  and  metal  sculpture.  Push  buttons 
are  mounted  on  selected  components  in  the  photos  and  on  selected 
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sculpted  components  as  well.  These  buttons  are  used  by  the  trainee 
to  indicate  the  location  of  components  that  are  identified  as  fault  sources 
during  malfunction  lessons. 


Insert  Figure  9  about  here 


Training  System  Software 

In  addition  to  standard  operating  system  software,  the  6883  MTS 
includes  trainer  common  modules  and  6883-specific  modTiles  which  pro¬ 
vide  flexibility  to  meet  changing  system  requirements. 

Trainer  common  modules.  The  trainer  common  modules  are 
the  building  block  subprograms  that  would  be  common  to  many  types  of 
trainers.  They  have  been  designed  to  be  inserted  into  a  variety  of  train¬ 
er  applications,  both  low  fidelity  and  high  fidelity  as  required.  These 
modules  include  the  following:  a)  training  system  controller,  b)  student 
procedure  monitor,  c)  instructor  monitor,  d)  student  test  routines, 
digital  voltmeter  and  oscilloscope  simulations,  and  self-test  diagnostics. 

6883-specific  modules.  The  6883-specific  modules  are  the 
building  block  subprograms  that  apply  primarily  to  the  current  6883  trainer 
and  to  its  unique  training  requirements.  The  building  block  approach  has 
been  used  here  to  allow  easy  modification,  resulting  from  changes  in 
trainer  requirements.  The  approach  allows  the  same  outline  to  be  used 
for  lesson  material  for  other  types  of  trainers. 

Instructional  Features 

In  this  portion  of  the  paper,  the  major  instructional  design  phil¬ 
osophy  will  be  discussed  along  with  a  review  of  the  courseware  structure, 
student  performance  measures,  and  student  and  instructor  feedback. 

Instructional  design  philosophy.  The  interactive  training 
philosophy  of  the  6883  system  is  depicted  in  Figure  10.  The  instructional 
design  of  the  6883  MTS  emphasizes  not  only  realistic  simulation  of  the 


Insert  Figure  10  about  here 
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actual  6883  system,  but  also  extensive  feedback,  monitoring,  cueing, 
and  guidance  of  trainee  actions  on  the  6883  MTS.  Here  the  computer 
replaces  the  instructor  as  the  master  overseer  of  the  system.  The 
computer  checks  the  accuracy  of  all  student  inputs,  determines  that 
technical  orders  are  being  followed  as  specified  in  the  instructional 
programs,  and  provides  appropriate  guidance  and  feedback  in  the  event 
of  an  error.  The  system  enables  the  trainee  to  conveniently  repeat 
a  sequence,  compress  time,  and  stop  the  program  if  so  desired. 

As  depicted  in  Figure  10,  the  student  has  a  direct  link  to  the  6883 
MTS.  In  a  typical  training  situation  which  utilizes  actual  equipment, 
the  student  is  required  to  check  with  the  instructor  prior  to  most  sys¬ 
tem  inputs.  In  the  simulator  training  environment,  the  instructor 
does  not  have  to  "hold  the  student  by  the  hand,  "  enabling  the  student 
to  work  at  his  own  pace.  Errors  made  by  a  student  are  recorded,  but 
can  easily  be  corrected  without  fear  of  injury  to  the  student  or  damage 
to  the  equipment.  Although  the  instructor  still  plays  an  active  role 
in  simulator  training,  the  computer  based  system  frees  up  the  instruc¬ 
tor  to  concentrate  on  the  remainder  of  the  class  which  is  waiting  to  use 
the  simulator.  Thus,  the  instructor's  monitoring  and  intervention 
requirements  are  dramtatically  less,  but  his  control  over  the  training 
environment  is  far  greater  than  in  the  conventional  situation.  The 
instructor  using  the  6883  MTS  can  choose  from  58  lessons  of  which  17 
are  LRU  malfunction  lessons  and  34  are  test  station  malfunctions.  The 
remaining  7  lessons  are  for  normal  procedures. 

Courseware  function  and  structure.  Courseware  may  be 
defined  as  the  computer-directed  presentation  of  instructional  material 
via  a  combination  of  media.  The  courseware  program  controls  the 
interaction  between  the  software  and  the  simulation  hardware  and 
between  both  of  these  and  the  trainee.  The  instructional  nn  terial 
includes  the  text  messages  presented  via  the  CRT  and  the  graphics 
presented  via  slides.  Courseware  is  essentially  a  set  of  computer 
programs  written  in  a  mnemonic  language  designed  for  the  6883  applica 
tion.  The  text  messages  called  for  by  the  program  are  written  in 
natural  English.  Because  the  6883  systan  is  a  procedures  trainer, 
the  courseware  completely  specifies  the  sequence  of  actions  expected 
of  the  trainee  and  the  information  that  the  trainee  receives  via  the 
CRT,  slide  projector,  and  the  test  station  displays.  The  courseware 
provides  for  prompting  and  feedback  where  necessary.  For  example, 
because  every  trainee  action  is  monitored,  any  incorrect  switch  or 
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control  setting  can  immediately  be  called  to  the  trainee's  attention. 

Since  the  structure  of  courseware  ia  modular  and  hierarchical,  lessons 
can  be  easily  modified  or  rearranged. 

Student  performance  measures.  The  6883  system  is 
designed  as  a  closed-loop  device.  The  actions  and  responses  of  trainees 
are  continually  monitored.  The  courseware  calls  out  the  expected 
action  or  answer  at  each  point  necessary  and  if  the  correct  response 
fails  to  occur,an  error  branch  is  invoked.  The  system  accumulates 
six  error  types  and  two  auxiliary  measures: 

Critical  or  safety  error 

Fault  detect  error 

Procedural  error 

Keyboard  (CRT)  error 

Component  location  error 

Switch/control  setting  configuration  error 
System  helps 
Student  helps 

Simulator  status  panel.  The  simulator  status  panel  located 
in  Bay  3  of  the  simulated  test  station  is  a  trainer -unique  panel  that 
provides  information  about  the  state  of  the  simulator  at  any  given  time. 
This  panel  is  not  found  on  the  actual  6883  test  station;  it  is  present  for 
training  purposes  only.  The  panel  contains  a  number  of  push  buttons 
and  indicators  designed  to  aid  in  performing  the  lessons.  Figure  11 
shows  this  special  purpose  panel. 


Insert  Figure  11  about  here 
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Instructor  aids.  When  any  type  of  error  occurs,  a  status 
log  is  automatically  displayed  at  the  instructor  station  CRT.  Figure 
12  shows  the  format  in  which  the  performance  information  is  presented. 
The  instructor  may  obtain  this  display  on  demand  from  the  system  and 
may  request  a  hard  copy  of  the  status  log. 


Insert  Figure  12  about  here 


System  Design  Features 

Hardware 

The  visual  simulation  required  for  panels  and  the  three-dimensional 
Simulation  needed  for  printed  circuit  cards  led  to  application  of  two  tech¬ 
nologies  that  saved  considerable  time  and  money  in  production. 

Metal  photos.  A  metal  photo  is  a  high-resolution  photograph 
embedded  in  the  surface  of  a  metal  pla.te.  Photographs  are  taken  of  the 
actual  equipment  as  the  first  step  in  the  development  of  metal  photos. 

Next,  photographic  half-tone  negatives  are  produced  and  a  photo-sensi¬ 
tive  aluminum  plate  is  then  exposed  to  the  negative  in  the  same  manner 
as  in  producing  a  standard  black-and-white  photographic  print.  The  re¬ 
sultant  high- re  solution  image  (1000  lines/mm)  is  sealed  under  an  ano¬ 
dized,  clear,  sapphire-hard  surface  that  protects  the  image  from  scratch¬ 
ing,  fading,  peeling,  and  chipping.  Normal  metalworking  processes 
such  as  bending,  cutting,  or  installing  components  may  be  used  on  these 
panels.  The  metal  photo  panels  thus  provide  the  durability  of  an  alumi¬ 
num  panel  with  the  low-cost  of  a  photograph.  The  use  of  this  technology 
avoids  the  cost  of  the  artwork  and  engraving  associated  with  actual 
panel  production. 

Circuit  cards.  The  simiilated  circuit  cards  were  produced 
by  laminating  color  photographs  onto  a  rigid  substrate,  an  aluminum 
alloy  sheet  for  all  cards  except  those  in  the  FCS  drawer.  For  the  FCS 
cards  a  glass/epoxy  substrate  was  used  to  electrically  insulate  their 
edge  connectors.  The  color  photographs  are  embedded  in  plastic  which 
both  attaches  them  to  the  substrate  and  provides  a  durable  surface. 

The  cards  for  the  DA  TAG  drawer  have  simulated  transistor  cans  bonded 
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to  their  surface.  The  color  photographs  of  the  actual  circuit  cards 
provide  a  high  degree  of  fidelity  with  low  cost.  Since  actual  circuit 
cards  are  quite  expensive,  this  technique  resulted  in  cost  savings  of 
75  to  80  percent. 

Software 

The  top-down,  modular  approach  to  software  design  proved  to 
be  the  best  method  for  achieving  reliable  and  manageable  programs 
for  the  training  requirement.  A  top-down,  modular  approach  using 
hierarchy  plus  input-processing-output  reports  allowed  a  more  effi¬ 
cient  use  of  programmer  talent,  provided  visible  levels  of  responsi¬ 
bility,  yielded  a  definite  software  architecture  easily  checked  for 
consistency  (due  to  well-defined  software  interfaces),  and  aided  the 
system  integration  and  verification  activities. 

The  top-down  modular  approach  assured  compliance  with  train¬ 
ing  requirements  by  a  check-off  method.  That  is,  each  computer 
program  component  was  divided  into  functional  portions  and  each 
module's  output,  function  and  input  requirements  were  defined  and 
validated.  This  close  relationship  between  functional  requirements 
and  software  module  allowed  rapid  cross-checking  by  nonprogramming 
personnel.  The  use  of  an  informal,  structured  program  design  language 
assisted  the  Air  Force  technical  personnel  in  checking  compliance  with 
requirements. 

Research  Test  Bed 

In  addition  to  designing  and  developing  the  6883  MTS  to  demon¬ 
strate  the  feasibility  of  simulation  technology  in  an  operational  training 
environment,  iFHRL  plans  to  use  the  6883  MTS  as  a  major  test  bed  to 
collect  baseline  information  and  conduct  research  in  several  areas  that 
should  significantly  impact  future  maintenance  simulator  procurements. 

Research  Issues 

Probably  the  most  important  question  to  ask  relative  to  the 
effectiveness  of  any  training  device  is,  "What  is  the  transfer  of  train¬ 
ing  to  the  actual  job  situation?  "  While  this  question  is  of  central 
importance,  attempts  to  answer  it  in  previews  research  have  produced 
inconclusive  results.  AFHRL  is  currently  attempting  to  answer  this 
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question  using  a  carefully  designed  study  incorporating  both  qualitative 
and  quantitative  approaches  to  determine  the  training  -  and  cost 
effectiveness  of  the  6883  MTS;  included  within  this  study  will  be  a 
trainee,  field  follow-up  to  assess  transfer  of  training. 

Other  research  questions  which  will  be  addressed  in  the  6883 
test  bed  environment  include  a)  the  level  of  fidelity  (realism)  required 
in  maintenance  simulators  (this  question  has  tremendous  cost  implica¬ 
tions),  b)  the  differential  effectiveness  of  static  versus  dynamic  display 
media,  and  c)  modularity  requirements  in  hardware  and  software. 
Research  is  currently  underway  to  develop  a  two-dimensional  simulation 
of  the  6883  MTS.  This  device  should  cost  approximately  50  percent  of 
the  6883  MTS  costs  and  will  enable  AFHRL  to  systematically  investigate 
varying  levels  of  fidelity  in  the  6883  environment  and  also  compare 
different  hardware  and  software  simulation  techniques.  The  above 
research,  coupled  with  R&D  in  even  less  expensive  techniques  such 
as  computer  graphics  and  paper  and  pencil  approaches,  should  provide 
the  Air  Force  with  answers  to  many  of  the  R&D  issues  discussed  above. 

Coupling  R&D  with  Operational  Requirements. 

Results  of  research  described  above,  along  with  othei:  R&D  con¬ 
ducted  in  AFHRL  Project  2361,  "Simulation  for  Maintenance  Training," 
will  be  incorporated  into  a  comprehensive  effort  to  provide  the  Air 
Force  with  new  specifications,  guides,  handbooks,  and  revised  Military 
Standards  specifically  tailored  to  the  design  and  procurement  of  new 
technology  maintenance  simulators. 
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Figure  1.  6883  Converter/Flight 
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Figure  4.  6883  Maintenance  Training  System  Hardware  Block  Diagram 


Lgure  5.  6883  MTS  Instructor 
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Figure  6.  6883  MTS  Student  Station 
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Figure  8.  Panel-by-Panel  Summary  of  Simulation 
Levels  Required  Across  Test  Station 
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Figure  19.  Interactive  Computer  Based  Training  in  the  6883  Maintenance  Training  System 
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Figure  12.  Status  Log  Display 
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Psychomotor/Perceptual  Measures  for  the  Selection  of 
Pilot  Trainees 


Abstract 


Measures  of  psychojnotor  and  perceptual  abilities  have  been  used 
for  the  selection  of  personnel  for  flying  training  since  World  War  II. 
Early  measures  showed  good  validity  for  the  selection  of .pilots; 
however,  the  operational  use  of  those  measures  was  discontinued  in  the 
early  1950’ s — mainly  due  to  calibration  and  maintenance  difficulties 
with  the  electromechanical  test  devices. 

Recently,  the  Personnel  Research  Division  has  Investigated  the 
use  of  psychomotor  and  perceptual  measures  obtained  from  modern,  solid- 
state  electronic  devices.  These  measures  were  shown  to  be  valid 
predictors  of  pilot  training  success.  Additionally,  measures  obtained 
from  flight  simulator  performance  have  shown  to  be  good  predictors 
of  training  success. 

Future  efforts  will  be  directed  at  the  identification  and  assess¬ 
ment  of  other  measures  of  psychomotor  and  perceptual  abilities  that 
may  be  related  to  performance  in  flying  training.  Further  research 
will  also  be  conducted  to  determine  if  low-fidelity,  "desk-top"  flight 
simulators  can  be  used  for  the  selection  of  pilot  trainees. 


1742 


Introduction 


Flying  training  is  expensive.  On  a  per  Individual  basis,  it  is 
probably  one  of  the  most  costly  of  the  training  programs  conducted  by 
the  military.  For  that  reason,  a  great  deal  of  attention  has  been  given 
to  the  problem  of  selecting  personnel  who  have  the  best  chance  of  succeed¬ 
ing  in  pilot  training~the  average  cost  of  each  person  who  attrits  from 
flying  training  in  the  Air  Force  Undergraduate  Pilot  Training  (UPT)  is 
now  well  over  $20,000.  With  a  training  flow  of,  say,  2,000  pilots  per 
year  and  an  attrition  rate  of  20%,  this  means  a  loss  of  over  $8,000,000 
per  year. 

The  reduction  of  attrition  from  UPT,  particularly  through  the 
development  of  improved  procedures  for  the  selection  of  trainees,  is 
therefore  a  matter  of  considerable  concern  to  the  Air  Force.  As  early 
as  World  War  I,  attention  was  being  directed  to  the  development  of  pro¬ 
cedures  for  the  selection  of  pilot  trainees.  In  1912,  instructions  were 
published  for  the  first  medical  examination  for  flying,  which  was  a 
requirement  for  entry  into  flight  training  (Passey  &  McLaurin,  1966). 

While  the  possible  contributions  which  psychological  testing  could 
maVp  to  the  selection  of  pilots  were  recognized  during  this  period,  little 
work  was  accomplished  until  immediately  prior  to  World  War  II .  At  that 
time  organizations  were  established  within  both  the  Army  Air  Force  and 
the  Navy  containing  specialists  in  aviation  psychology.  One  of  their 
primary  concerns  was  the  selection  and  classification  of  flying  personnel . 

In  addition  to  the  development  of  a  battery  of  paper-and-pencil 
tests  for  the  selection  of  flying  personnel,  many  tests  of  psychomotor/ 
perceptual  abilities  were  developed,  principally  by  personnel  of  the  Army 
Air  Corps. 

Figure  1  shows  the  apparatus  tests  used  by  the  Army  Air  Corps  test¬ 
ing  battery  in  February  1942. 

Figure  2  shows  the  apparatus  tests  used  in  June  1945.  These  tests 
are  described  in  detail  by  Melton  (1947).  Not  all  of  these  tests  were 
ased  for  the  selection  of  pilot  trainees.  The  Finger  Dexterity  test, 
for  example ,  was  used  only  for  the  selection  of  bombardiers . 

One  of  the  better  known  of  these  tests  is  the  SAM  Complex  Coordination 
test.  Figure  3  shows  the  apparatus  used  in  that  test.  As  may  be  seen 
from  the  figure,  the  apparatus  was  comprised  of  a  control  stick,  rudder 
bar,  and  an  array  of  lights.  The  examinee  would  manipulate  the  control 
stick  and  rudder  so  as  to  line  up  pairs  of  lights  in  the  array.  Measures 
obtained  from  this  device — the  number  of  matches  obtained  in  a  given  time 
were  found  to  reliably  predict  later  performance  in  pilot  training 
(correlation  on  the  order  of  .25). 
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While  this  test  and  some  of  the  others  shown  in  Figure  2  could 
contribute  to  the  identification  of  potentially  successful  pilot 
trainees,  their  use  in  the  operational  screening  process  was  discontinued 
in  1955.  This  came  about  as  a  result  of  the  dispersal  of  the  applicants 
to  be  tested  and  also  because  of  the  logistic  problems  involved  in 
apparatus  testing — especially  the  difficulties  inherent  in  the  cali¬ 
bration  and  maintenance  of  devices  which  used  the  electro-mechanical 
technology  of  that  period. 

From  that  time  until  very  recently,  little  attention  was  given  to 
the  use  of  psychomotor /perceptual  measures  for  the  selection  of  personnel 
for  entry  into  Air  Force  UPT. 

Laboratory  Tests 

Two  developments  have  brought  about  a  reawakening  of  interest  in 
the  use  of  psychomotor /perceptual  or,  more  generally,  apparatus  tests, 
for  the  selection  of  pilot  trainees;  (1)  the  ready  availability  of 
reliable,  solid-state  circuitry  for  use  in  testing  devices;  and  (2)  an 
apparent  peaking  out  of  validity  for  paper-and-pencil  tests.  The  use 
of  solid-state  devices  for  testing  would  mean  that  many  of  the  calibration 
and  maintenance  difficulties  that  beset  the  World  War  II  electro-mechanical 
devices  could  be  eliminated  and,  possibly,  reliable  test  devices  produced 
would  be  light,  contact,  and  relatively  inexpensive. 

Experience  from  the  research  conducted  during  World  War  II  had 
shown  that  psychbmotor/perceptual  tests,  for  the  most  part,  had  little 
overlap  with  paper-and-pencil  tests  and  that  useful  increases  in 
validity  could  be  effected  through  their  combination  with  paper-and-pencil 
tests  in  a  selection  battery.  Continual  revisions  to  the  paper-and-pencil 
selection  batteries  had  resulted  in  the  development  of  ah  instrument, 
the  Pilot  Composite  of  the  Air  Force  Officer  Qualifying  Test,  that  could 
reliably  predict  performance  in  pilot  training  to  an  acceptable  degree 
(correlations  on  the  order  of  .40  -  .50);  however,  in  a  group,  paper- 
and-pencil  testing  situation,  the  range  of  abilities  that  may  be  assessed 
is  limited,  and  it  becomes  difficult  to  increase  the  validity  of  the 
paper-and-pencil  battery . 

As  a  consequence,  the  Personnel  Research  Division  landertook  the 
study  of  new  procedures  for  the  measurement  of  psychomOtor/perceptual 
abilities.  Under  contract,  a  laboratory  facility  was  constructed  and  two 
tests  of  psychomotor/perceptual  ability  developed.  This  laboratory 
system  consisted  of  a  PDP-8/L  minicomputer  interfaced  to  the  two  test 
stations  shown  in  Figure  4 . 

As  shown  in  Figure  4,  each  test  station  consisted  of  a  direct-view 
cathode  ray  tube  display,  two  hand  controllers,  a  large,  floor-mounted 
joystick,  and  a  rudder  bar.  The  use  of  a  minicomputer  based  system  was 
Selected  because  of  the  flexibility  which  this  system  afforded  in  the 
alteration  of  testing  procedures  and  the  development  of  new  tests. 
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The  two  tests  which  were  initially  implemented  on  this  system  were 
modeled  after  thd  psychological  test  characteristics  of  their  World  War  II 
namesakes:  Twovkand  Coordination  and  Complex  Coordination  (Sanders, 
Valentine,  &  McGrevy,  1971). 

Shown  in  Figure  5  are  the  displays  used  in  both  of  these  tests. 

In  the  Two-Hand  Coordination  test,  the  examinee  uses  the  two 
table-mounted  joysticks,  one  in  each  hand,  to  control  the  position  of  an 
X-shaped  cursor  on  the  screen.  The  left  stick  controls  vertical  movement, 
the  right  stick  horizontal  movement  of  the  cursor.  Instructions  to  the 
examinee  require  that  he' maintain  the  position  of  the  X  as  close  as  he 
can  to  a  triangular  target  which  moves  in  a  circular  path  at  varying 
speeds.  This  test  is  scored  by  summing  the  absolute  displacements  from 
the  cursor  to  the  target  in  the  X  and  Y  axes  over  some  specified  time 
interval.  Typically,  a  1-minute  interval  was  selected.  The  test  con¬ 
sisted  of  3  minutes  of  directed  practice  followed  by  5  minutes  during 
\diich  performance  measures  were  recorded;  thus,  ten  measures  were 
obtained — one  per  axis  per  minute. 

In  the  Complex  Coordination  test,  the  examinee  is  required  to 
manipulate  the  floor-mounted  joystick  to  control  the  movement  of  an 
X-shaped  cursor  while  at  the  same  time  using  both  feet  on  rudder  pedals 
which  control  a  short  vertical  line  which  hovers  near  the  bottom  of  the 
screen.  Instructions  to  the  examinees  require  that  he  hold  the  cursor 
stationary  at  the  intersection  of  the  fixed  vertical  and  horizontal 
line  of  dots  while  keeping  the  vertical  line  aligned  with  the  vertical 
line  of  dots.  Error  scores,  that  is,  summed  absolute  displacements 
from  the  cursor  to  the  intersection,  are  recorded  separately  for  the  X 
and  Y  axes  for  each  minute  of  the  test.  In  addition,  the  error  score 
for  the  vertical  line  under  control  of  the  rudder  bar  is  recorded  for 
each  minute  of  the  test.  Like  the  Two-Hand  Coordination  test,  there  is 
a  3-minute  directed  practice  period  followed  by  5  minutes  of  testing. 

The  measures  taken  from  both  of  these  tests  were  found  to  success¬ 
fully  predict  later  performance  in  pilot  training.  Figure  6  shows  the 
mean  scores  obtained  by  three  groups  on  the  Complex  Coordination  test. 

It  may  be  seen  from  this  figure  that  the  group  with  the  highest  score, 
and  hence  the  poorest  performance  since  these  are  error  scores,  consisted 
of  those  individuals  who  were  eliminated  from  Undergraduate  Pilot  Training 
due  to  Flying  Training  Deficiency  (FTD) .  The  group  with  the  lowest  score, 
and  hence  the  best  performance,  consisted  of  those  individuals  who  gradu¬ 
ated  from  pilot  training  (McGrevy  &  Valentine,  1974). 

jjig  ]”esults  obtained  from  the  Tv7o— Hand  Coordination  test  are  similar, 
although  not  so  dramatic,  as  those  displayed  in  Figure  6.  In  general, 
the  Complex  Coordination  test  is  superior  to  the  Two-Hand  Coordination 
test  in  the  selection  of  pilot  trainees.  These  results  have  been  replicated 
with  approximately  the  same  outcomes  in  two  follow— on  studies  (cf.  Hunter 
&  Thompson,  1978). 
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Portable  Test  Devices 


As  a  result  of  the  success  obtained  from  the  laboratory  versions  of 
these  two  tests,  it  was  decided  to  develop  a  portable,  self-contained 
device  for  the  administration  of  these  tests  at  field  locations  such  as 
at  the  Armed  Forces  Examining  and  Entrance  Stations  (AFEES)  or  Reserve 
Officer  Training  Corps  detachments;  the  concept  was  to  take  the  test  to  the 
examinees  rather  than  have  examinees  brought  to  a  central  screening  point. 

Like  the  laboratory  verison,  the  portable  test  device  has  two  table- 
mounted  joysticks,  a  floor-mounted  joystick,  rudder-bar,  and  a  cathode 
ray  tube  display.  The  instructions  to  the  examinees  and  the  displays 
used  during  the  tests  are  also  virtually  Identical  to  those  used  in  the 
laboratory  version. 

This  unit  provides  for  completely  automated  presentation  of 
instructions,  via  a  cassette  tape,  and  automatic  testing  and  scoring. 

As  before,  scores  are  summed  absolute  deviations  from  the  cursors  to  the 
target  points;  however,  the  time  interval  over  which  scores  are  recorded 
has  been  altered.  Analyses  of  data  taken  from  the  laboratory  versions 
of  the  two  tests  indicated  that  most  of  the  useful  information  was 
obtained  from  performance  in  the  final  2  minutes  of  the  test  cycle. 
Therefore,  the  portable  device  records  and  displays  only  thei  summed 
error  scores  from  the  last  2  minutes  of  each  test.  Separate  scores  are 
obtained  for  each  of  the  axes,  however,  so  that  for  the  Complex 
Coordination  test  three  scores  are  obtained  while  for  the  Two-Hand 
Coordination  test  only  two  scores  are  obtained. 

Iiiitial  field  trials  conducted  both  at  Lackland  Air  Force  Base  and 
at  various  ROTC  detachments  have  shown  this  device  to  be  rugged  and 
reliable  and  easy  for  inexperienced  personnel  to  operate.  Data  is  now 
being  collected  using  these  devices  on  personnel  entering  pilot  training 
at  Williams  AFB,  Arizona,  and  on  cadets  at  the  Air  Force  Academy  who 
are  slated  to  enter  pilot  training.  This  data  will  be  used  to  further 
validate  the  use  of  these  tests  for  the  selection  of  personnel  for  flying 
training  and  to  obtain  additional  feedback  from  field  personnel  on  ways 
in  which  the  design  and  operation  of  the  devices  may  be  improved.  This 
information  will  then  be  used  to  improve  the  design  and  characteristics 
of  production  versions  of  the  devices  which  may  be  used  in  the  operational 
selection  of  pilot  trainees. 

Learning  Ability 

In  addition  to  the  approach  taken  in  the  design  of  the  tests  described 
above — that  is ,  the  measurement  of  relatively  pure  basic  abilities—another 
approach  subsumed  under  the  title  of  psychomotor/perceptual  measures 
Involves  the  measurement  of  an  individual’s  capacity  to  learn  a  task  or 
complex  series  of  tasks—in  this  case  the  task  of  flying  an  aircraft. 
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It  has  long  been  noted  that  the  best  way  to  select  an  individual  for 
a  position  is  to  simply  put  him/her  in  that  position  and  observe  his/her 
performance.  Thus,  the  best  predictor  of  success  in  flying  training  is 
flying  training.  This  is  the  notion  that  underlies  the  use  of  the  light- 
plane  (T-41)  screening  program  used  by  the  Air  Force.  This  process 
measures  the  ability  of  the  individual  to  learn  the  same  or  very  nearly 
the  same  tasks  that  will  be  later  required  in  the  training  program;  the 
closer  the  similarity  between  the  initial  or  test  tasks  and  the  final 
task,  the  higher  the  validity  should  be  of  the  procedure. 

To  investigate  that  approach  outside  of  an  actual  aircraft  cockpit, 
the  Personnel  Research  Division,  through  a  contract  with  McDonnell  Douglas 
Corporation,  developed  the  Automated  Pilot  Aptitude  Measurement  System 
(APAMS) .  The  hardware  of  the  APAMS  consists  of  two  modified  Singer-Link 
GAT-1  light  aircraft  simulators  interfaced  to  a  small  minicomputer 
(Varian  620-f/100).  Feedback  to  the  examinees  was  provided  by  a  cathode- 
ray  tube  display  motmted  above  the  instrument  panel  and  by  a  Votrax  voice 
synthesizer.  Instructions  to  the  examinees  were  given  via  a  Bell  and 
Howell  filmstrip  system  mounted  to  the  left  of  the  examinee  in  the  cock¬ 
pit  (Long  &  Varney,  1975). 

The  Instructional  system  used  in  this  study  assumed  no  prior  flight 
experience  on  the  part  of  the  examinees.  Therefore,  all  examinees  received 
instruction  in  the  purpose  and  use  of  controls  (e.g. ,  throttle,  control 
wheel)  and  instruments  before  receiving  instruction  in  how  to  perform 
flight  maneuvers.  The  syllabus  of  Instruction  required  approximately 
5  hours,  divided  into  five  session  of  about  1  hour  each,  spread  over  10 
days . 

After  learning  the  basic  functions  of  the  controls  and  instruments, 
the  examinees  learned  how  to  fly  the  simulator  in  straight-and-level 
flight,  how  to  perform  turns  and  descents,  and,  by  the  end  of  the  5th 
hour,  were  performing  take-offs  and  landings  and  flying  an  airport  traffic 
pattern. 

During  each  stage  of  the  learning  process,  performance  was  measured 
automatically  by  the  computer.  Measures  were  in  terms  of  deviations 
from  the  command  or  ideal  state  of  flight' parameters  such  as. heading, 
altitude,  and  airspeed.  This  process  generated  190  measures  of  per¬ 
formance  at  differing  stages  of  learning.  Factor  analysis  and  other 
data  reduction  procedures  eventually  reduced  this  number  to  approximately 
six  scores  which  could  parsimoniously  describe  the  examinees  performance. 

From  these  analyses,  it  was  found  that  measures  of  Heading,  Bank, 
and  Altitude  control  could  reliably  predict  later  performance  in  pilot 
training.  In  fact,  these  measures  were  superior  to  either  the  paper- 
and-pencil  or  psychomotor  coordination  tests  in  the  prediction  of  outcomes 
in  pilot  training.  (Zero  order  correlations  on  the  order  of  .20-. 35). 
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Sianmarv  and  Conclusions 

These  studies  have  shown  that  measures  of  psychomotor/perceptual 
abilities  may  be  reliably  assessed  and  that  such  measures  are  valid  pre¬ 
dictors  of  later  performance  in  pilot  trainings  Furthermore,  these 
measures  are  relatively  independent  of  those  abilities  assessed  through 
the  use  of  conventional  paper-and-pencil  tests.  This  means  that 
psychomotor/perceptual  measures  may  make  significant  contributions  to 
the  existing  selection  procedures  which,  except  for  the  physical  examina¬ 
tion,  rely  exclusively  on  conventional  paper-and-pencil  tests. 

The  measures  considered  thus  far  are,  of  course,  by  no  means  the 
only  ones  that  may  prove  useful.  Indeed,  the  present  research  has 
barely  begun  to  identify  those  psychomotor/perceptual  abilities  that  may 
be  related  to  success  and  failure  in  pilot  training.  Other  abilities 
measured  during  World  War  II,  such  as  choice  reaction  time,  and  more 
recently  identified  measures  such  as  information  processing  ability  as 
measured  under  high  load  conditions,  may  also  prove  relevant. 

The  future  research  to  be  conducted  in  this  area  will  seek  to  identify 
more  of  these  abilities  that  are  related  to  pilot  training  outcomes  and 
to  develop  reliable  and  inexpensive  Instruments  for  their  assessment. 

In  addition,  procedures  for  the  measurement  of  abilities  already  identified 
will  be  Improved. 

In  the  laboratory,  the  relation  between  learning  ability,  as  measured 
by  the  APAMS,  and  success  in  pilot  training  has  been  established.  However, 
the  devices  used  in  the  laboratory  are  totally  unsuitable  for  use  in  an 
operational  selection  system.  The  next  step  in  this  area  will  be  to 
develop  an  inexpensive,  portable  device  for  the  assessment  of  learning 
ability,  and  this  will  be  accomplished  in  the  near  future. 

The  design  of  an  effective  pilot  selection  system,  like  the  design 
of  a  weapons  system,  is,  in  the  end,  directed  by  cost-benefit  relation¬ 
ships.  It  is  not  sufficient  to  design  a  valid  selection  test  without 
at  the  same  time  considering  the  eventual  acquisition  and  operation  costs 
of  that  test  in  comparison  with  the  savings  to  be  realized  through 
decreased  attrition  or  improved  personnel  effectiveness . 

These  factors  will  continue  to  be  considered  during  the  design  and 
development  of  new  psychomotor/perceptual  tests  so  that  these  instru¬ 
ments  can  make  a  positive  impact  upon  the  reduction  of  training  costs 
and  the  Improvement  of  personnel  utilization  in  the  Air  Force. 
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Figure  2. 
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Abstract 


This  paper  presents  the  case  for  using  simulators  and 
other  new  training  technologies  rather  than  costly  actual 
equipment  to  teach  hands-on  troubleshooting  skills  to 
maintenance  technicians.  The  likely  impact  of  these  newer 
synthetic  training  devices  on  fleet  readiness  and  cost 
parameters  is  discussed.  The  presentation  identifies 
the  danger  of  trying  to  solve  the  technical  training  prob¬ 
lem  with  one-sided  solutions  which  do  not  take  advantage 
of  modern  training  techniques. 
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Introduction 


We  have  some  good  news,  and  some  bad  news.  The  good 
news  is  that  the  high  cost  of  maintenance^ for  DoD  equipment 
is  not  the  major  problem  to  be  discussed  in  this  paper.  Yes, 
you*ve  heard  the  figures  that  maintenance  accounts  for  twenty 
to  twenty-five  percent  of  the  total  annual  DoD  budget. 

(Rowan,  1973),  and  you  may  have  even  seen  the  data  to  the 
effect  that  it  costs  DoD  one  hundred  million  dollars  per 
year  just  to  train  electronics  technicians  (Bruns ,  1975) . 

One  example  that  seems  relevant  here  is  shown  in  Figure  1 
where  Repair  Labor,  as  a  life-cycle  cost  element  in  the 
A-7D  aircraft  weapon  system,  was  essentially  equal  to  the 
initial  purchase  price  of  the  system,  plus  the  cost  of 
operating  it,  including  pilot  salaries.  But  we  re  not  going 
to  dwell  on  the  costs  of  maintenance!  per  se. 

You  can  get  a  hint  of  the  nature  of  the  bad  news  from 
the  fact  that  even  if  we  allocated  twice  the  money  we  now 
spend  on  troubleshooting  and  repairing  our ^Complicated^ 
weapons  systems,  we  probably  wouldn't  use  it  any  more  judi¬ 
ciously  than  we  now  utilize  our  present  budget.  The  bad  news 
ladies  and  gentlemen,  is  that  the  maior  problem  facing  o 
today  is  one  of  readiness.  The  discomtorting  fact  is  that 
if  we  cannot  improve  our  readiness  posture  beyond  what  it 
is  today,  we  simply  are  not  going  to  survive  long  enough 
to  worry  about  the  finer  points  of  Repair  Labor  cost. 

Let's  take  a  brief  look  at  our  readiness  posture.  The 
continuing  Soviet  buildup,  and  simultaneous  waning  of  Ameri¬ 
can  military  superiority  has  not  escaped  notice,  with  the 
alarm  even  being  sounded  in  the  respected  journal  Foreign 
Affairs  (Gray,  1978;  Head,  1978).  Current  figures  prepared 
for  the  Chief  of  Naval  Operations  (Understanding  Soviet  ^ 
Naval  Developments,  1978)  show  the  results  of  recent  Soviet 
Naval  increments  during  a  period  in  which  the  size  or  the 
U.S.  Navy  decreased  by  over  50  percent.  Figure  2  tells  the 
story:  the  Soviet  Navy  is  the  world's  largest  in  terms  of 

numbers  of  ships,  outranking  us  in  overall  tonnage  and 
outnumbering  us  in  every  category  except  aircraft  carriers. 
They  are  embarking  on  a  carrier-building  enterprise  which 
will  put  them  essentially  in  control  of  the  seas. 

But  how  about  missilery?  Surely  our  technological 
superiority"  will  sparkle  in  this  category.  Unfortunately, 
Gray  (1978)  has  predicted  that  by  the  ear ly-to-mid- 1980  s. 
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we  may  have  to  accept  that  all  or  nearly  all  of  our  silo- 
housed  missile  force  would  be  destroyed  hy  a  Soviet  first 
nuclear  strike. 

Another  surprising  fact  is  shown  in  Figure  3,  which 
illustrates  how  Soviet  R&D  expenditures  have  grown  until 
their  outlays  exceed  those  of  the  U.S.,  and  how  those  efforts 
have  led  them  to  be  able  to  outspend  us  in  weapons  procure¬ 
ment  and  military  construction  by  two  to  one  in  1976. 

Figure  4  compares  U.S.  and  Soviet  production  rates  for 
several  military  categories.  These  figures  show  them  out¬ 
producing  us  eight  to  one  in  artillery,  six  to  one  in  tanks, 
three  to  one  in  infantry  fighting  vehicles,  two  to  one  in 
tactical  aircraft,  and  with  slight  advantages  in  helicopters 
and  anti-tank  missiles. 


Are  We  Ready? 

While  Americans  are  justifiably  proud  of  the  sophistica¬ 
tion  of  their  weapon  systems,  that  very  sophistication  can 
have  serious  consequences  for  readiness.  In  an  interesting 
perspective  on  U.S.  and  Soviet  R&D  systems,  Head  (1978) 
presents  convincing  arguments  against  any  fond  hope  that 
Soviet  weaponry  is  likely  to  be  in  a  worse  state  of  readiness 
than  ours.  For  example,  Soviet  weapons  designers  may  have 
the  materials  and  training  to  develop  more  advanced  designs, 
but  their  efforts  are  circumscribed  by  several  constraints. 
First,  they  must  use  centrally-approved  designers'  handbooks 
that  specify  research  results,  an  approved  list  of  structures, 
design  forms,  components,  materials,  and  manufacturing  tech¬ 
niques.  They  are  also  restrained  by  lack  of  sophistication 
in  production  technology,  low  technical  level  of  Soviet 
troops,  and  a  doctrine  that  military  capabilities  are  en¬ 
hanced  more  by  large  numbers  of  deployed  weapons  with  modest 
individual  characteristics  than  by  smaller  niambers  of  higher 
quality  weapons .  These  factors  produce  a  conservative 
design  simplicity,  interchangeable  parts,  and  evolutionary 
growth  (Alexander,  1976). 

By  contrast,  U.S.  weapons  development  is  oriented 
toward  high  performance.  U.S.  military  requirements  call 
for  high  performance  and  low  attrition  rate,  and  U.S. 
industrial  contractors  respond  with  proposals  for  revolu¬ 
tionary  developments,  new  subsystems,  and  sophisticated 
design.  This  design  approach  has  often  produced  overly 
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complex,  expensive,  and  less  reliable  Systems.  As  you  would 
expect,  the  sophistication  of  U.S.  weapons  systems  has 
tended  to  generate  higher  maintenance  manpower  and  training 
costs  and  higher  support  costs  (Head,  1978). 

The  readiness  implications  seem  clear.  Put  it  this 
way:  You  might  feel  pretty  confident  if  you  could  have  an 
F-14  aircraft  armed  to  the  teeth  assigned  exclusively  to 
protect  your  home.  But  consider  what  happens  on  the  day 
your  F-14  has  its  Constant- Speed  Drive  in  the  shop  for  repairs 
and  someone  comes  down  your  street  driving  a  tank  with  a 
manual  transmission,  a  manual,  lever- type  steering  mechanism, 
and  a  40-year  old  engine  design.  The  Soviet  T-62  tank  is  just 
such  a  tank.  Uncomplicated  and  unsophisticated?  Perhaps. 

But  you  can  bet  if  they  want  to  use  it,  it  will  be  ready! 

While  there  are  data  to  support  the  above  sensationalist 
scare  tactic,  they  are  very  preliminary,  and  we  do  not  want 
to  designate  the  exact  type  of  weapon  system  involved,  nor 
the  exact  findings.  And,  of  course,  any  open  discussion  of 
American  readiness  factors  would  overlook  the  sensitive 
nature  of  such  topics  to  our  national  defense.  Let  us  just 
say  that  "a  recent  study"  shows  that  the  number  of  weapon 
systems  cited  in  our  favor  at  the  SALT  tasks  in  Geneva  are 
an  over-estimate  of  the  real  number  of  systems  remaining 
after  agedness,  mean-time-between-failure,  and  maintenance 
errors  have  all  taken  their  toll. 

In  efforts  such  as  the  "recent  study"  above,  we  can 
show  that  a  particular  group  of  aircraft  are  NOR  (Not  Opera¬ 
tionally  Ready)  due  to  specific  combinations  of  maintenance 
errors,  faulty  procedures,  and/or  poor  sparing  policies . 

And  we  can  simulate  the  possibilities  of  reducing  that  NOR 
figure  by  half  without  expending  over  one- tenth  of  the  cost 
of  adding  another  aircraft.  The  question  then  becomes  one 
of  achieving  that  reduction  using  purely  the  Human  Factors 
and  Human  Resources  approach.  In  the  section  below,  we 
want  to  lay  to  rest  some  current  and  proposed  approaches 
that  won*t  help  us  reach  that  objective. 


What  Won't  Work 

Hardware  Solutions.  "Hardware"  solutions  to  today's 
costly  maintenance  problems  are  suspect  from  the  start  be¬ 
cause  we  know  that  maintenance  and  readiness  both  rely 
heavily  on  people  systems.  In  the  Apollo  Program,  it  was 
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the  huinan.  astironaut  who  provided  the  manual  backup  in  many 
cases  to  provide  mission  reliability.  Manufacturers'  claims 
regarding  ATE  BITE,  and  "Smart"  instruments  notwithstanding. 
It  is  ultimately  the  human  technician  who  winds  up  trouble¬ 
shooting  the  really  tough  malfunctions.  Figure  5  summarizes 
results  of  one  study  where  the  promised  MTBF  was  in  every 
case  more  glowing  than  the  eventual  MTBF  in  the  real  world 
(Pyatt,  1972). 


Another  example  comes  from  the  largest  single  deploy¬ 
ment  of  ATE  (Automatic  Test  Equipment)  in  military  history 
the  Navy  VAST  system.  While  this  workhorse  has  accom¬ 
plished  much  of  its  mission,  it  has,  at  the  same  time, 
created  a  "vast"  number  of  gray-haired  technicians.  True 
to  the  Hardware-Solution  tradition,  VAST  was  supposed  to 
require  an  operator  and  technician  with  only  minimal  train¬ 
ing.  Unfortunately,  such  a  concept  presupposes  that  the 
program  will  be  perfect,  the  machine  will  always  operate 
properly,  and  documentation  associated  with  the  testing 
process  will  always  be  up-to-date  and  correct.  Experience 
has  shown  that  such  conditions  seldom  prevail  in  spite  of 
the  most  stringent  efforts.  In  the  end,  the  VAST  program 
has  had  tO: 


o  realign  training  to  include  advanced  operator  and 
intermediate  maintenance  courses,  including  off-line 
maintenance  procedures,  calibration,  self-test,  and 
in-depth  theory. 


o  acquire  supplemental  data  such  as  diagnostic  flow- 
>  string  lists,  test  diagrams,  and  program 
listings  to  provide  troubleshooting  data  when  the 
test  program  does  not  provide  the  right  answer. 

o  invent  training  for  a  new  kind  of  technician  called 
a  Test  Program  Set  Analyst,  whose  job  it  is  to  try 
to  make  sense  out  of  it  all. 

The  VAST  experience  is  generally  applicable  across 
several  types  of  ATE,  and  the  magnitude  of  the  resulting 
maintenance  readiness  problem  can  be  grasped  from  the  fact 
that  acquisitions  of  ATE  amount  to  500  million  dollars 
annually  in  the  Navy  and  700  million  in  the  Air  Force  (King 
and  Duva,  "Overview"  1978) .  ° 
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Minimize  training,  maximize  Job-Performance  Aiding. 

There  is  a  sizable  effort  mounting  in  DoD  to  replace  training 
with  Job-Performance  Aids  (JPAs)  which  tell  the  technician 
exactly  What  to  do  in  the  work  setting.  Experiments  have 
been  cited  where  high  school  graduates  first  receive  a  few 
hours  instruction  on  how  to  use  this  "cookbook"  approach  for 
checking  out  a  piece  of  equipment.  Their  performance  on 
checkout  of  an  actual  piece  of  equipment  is  then  compared 
with  that  of  a  group  of  seasoned  veterans .  For  the  parti¬ 
cular  tasks  studies,  the  veterans  did  no  better  than  the 
high-schoolers . 

Unfortunately,  the  minimized  training,  maximized  aiding 
approach  suffers  from  a  number  of  critical  drawbacks : 

(1)  Ignores  Typical  Navy  Environments .  Work  environ- 
ments  in  military  services  other  than  Navy  are  typically 
accessible  by  land  transport  and  re- supply;  thus  space  is 
not  the  critical  issue  that  it  is  in,  for  example,  the 
submarine  Navy  where  eight  tons  of  displacement  must  be 
provided  just  for  the  life  support  of  each  man  taken  aboard. 
The  latter  environment  cannot  tolerate  the  concept  of  several 
lesser-trained  individuals  backed  up  by  a  senior  technician. 
Each  man  must  be  a  professional,  and  he  must  even  be  crossed- 
trained  in  areas  not  orignially  his  own.  In  the  NAVAIR  case, 
the  cramped  quarters  of  a  carrier-based  aircraft  are  only 
slightly  relieved  when  the  airplane  lands  bn  the  deck  of  a 
floating  platform  which  has  its  own  re-supply  problems. 

The  space/re-supply  problem  has  important  implications 
for  maintenance  training  in  that  the  technician  must  be 
able  to  (a)  diagnose  and  repair  systems  down  to  the  component 
level,  a  requirement  that  rules  out  the  use  of  simple  check¬ 
lists  which  rely  primarily  on  a  diagnose-by-replacement 
strategy,  and  (b)  operate  without  the  voluminous  docximentation 
that  it  would  take  to  provide  step-by-step  instructions  for 
troubleshooting  complicated  systems  at  the  Intermediate  Level 
of  difficulty.  The  situation  is  somewhat  as  shown  in  Figure 
6  where  we  see  the  Navy  technician  having  a  sizable  system 
responsibility  with  only  limited  space  available  for  spares 
and/or  documentation.  As  a  result,  we  must  make  sure  he  has 
mentally  stored  sufficient  principles  and  hands-on  training 
to  cope  with  the  problems  that  he  undoubtedly  will  en¬ 
counter  in  the  Fleet. 
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(2)  Concentrates  on  Outmoded  Paper-Covered  Aiding. 

What  could  be  more  efficient  than  a  simple  sheet' b£  paper 
that  tells  the  technician  exactly  what  to  do?  First,  there 
is  no  assurance  that  that  sheet  will  prove  to  be  any  better 
than  the  standard  technical  manual  in  terms  of  its  update 
capability.  In  fact,  its  necessary  specificity  could  cause 
the  checklist  to  be  even  more  sensitive  to  update  require¬ 
ments  and  thereby  more  conducive  to  maintenance  error  than 
the  familiar  tech  manual  which  presents  only  general  system 
information. 

Aside  from  the  likelihood  that  paper -covered  JPAs  will 
exhibit  early  obsolescence  in  a  system's  life  cycle,  it  is 
inconceivable  that  a  pre-composed,  printed,  and  published 
JPA  can  be  written  for  troubleshooting  complex  systems  down 
to  the  component  level.  Our  disbelief  here  stems  from  the 
simple  possibility  that  a  malfunction  occurring  on  a  parti¬ 
cular  system  under  specified  conditions  may  never  have  occurred 
before  to  a  JPA  author.  In  other  words,  it  is  difficult  to 
conceive  of  a  step-by-step  guide  (ignoring  the  volume  of 
paper  that  would  result)  for  troubleshooting  individual 
malfunctions  of  a  complex  system,  when  even  it's  designer 
might  not  know  what  would  happen  under  many  circumstances. 

(3)  Mission  Success  Hinges  Upon  Only  a  Few  Individuals. 
The  concept  of  only  a  few  individuals  trained  to  a  professional 
level  on  a  weapon  system  (who  oversee  the  JPA-based  activi¬ 
ties  of  a  number  of  technicians)  sets  a  dangerous  precedent 

in  terms  of  tactical  warfare.  One  professional  JPA  Specialist 
in  remarking  on  the  ill-fated  rush  to  provide  FPJPAs  to  the 
Vietnamization  effort,  concludes  that  "The  studies  conducted 
to  date  reinforce  the  feasibility  of  integration  of  the  JPA 
approach  into  the  documentation  system,  but  not  replacement 
of  the  existing  system  with  JPAs  --an  intent  that  many 
erroneously  inferred  from  the  hastily-assembled  MIL-J-83302 . " 
(Joyce,  1975,  pg.  11). 

While  we  may  not  be  training  our  technicians  in  a  most 
cost-effective  and/or  training  effective  manner  using  time¬ 
worn  methods,  our  present  state  of  readiness  (or  unreadiness, 
as  the  case  may  be)  suggests  that  this  would  not  be  an  appro¬ 
priate^  time  to  cut  out  training  altogether.  As  the  saying 
goes,  "If  you  think  training  is  expensive,  try  ignorance." 
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Where  Do  We  Go  From  Here? 

One  interesting  figure  on  maintenance  errors  that  has 
been  cited  since  1973  pertaining  to  false  returns,  is 
that  for  all  of  DoD,  3079  of  the  units  that  are  returned 
as  being  faulty  are,  in  fact,  good  (Rowan,  1973).  It  is , 
of  course,  our  presumption  here  that  many  of  these  types 
of  errors  could  be  diminished  with  improved  maintenance 
training  both  in  the  classroom  and  on  the  job. 

In  calling  for  more  formal  classroom- type  training 
for  technicians,  Huffman  and  Rostker  (1976)  point  put  that 
formal  training  may  be  less  expensive  than  OJT,  and  they 
cite  the  study  by  Gay  (1974)  in  this  regard.  As  reviewed 
by  Huffman  and  Rostker,  Gay  showed  that  the  OJT  cost  for 
Aircraft  Maintenance  Specialists  in  the  Air  Force  was 
approximately  twice  that  of  their  technical  school  training, 
and  about  half  the  total  cost  of  the  Air  Force's  first- 
term  investment  in  the  Airman.  It  is  easy  to  underestimate 
the  cost  and  extent  of  OJT;  for  example,  most  of  us  at 
this  conference  are  learning  something  from  the  discussions, 
that  is,  we  are  learning  concepts  on-the-job. 

The  point  is  that  maintenance  and  its  logical  extension 
readiness,  both  need  a  fresh  approach  which  looks  at  the 
people  variables  in  the  entire  maintenance  pipeline.  What 
we  see  is  a  propitious  mix  of  the  media  represented  by  the 
bottommost  line  of  six  blocks  in  Figure  7.  In  other 
words,  we  believe  in  "traiding"  (Training  and  Aiding) 
the  technician  sO  that  he  is  of  both  immediate  and  long 
term-use  to  the  operating  unit.  We  propose  to  do  this  as 
follows ; 

(1)  Bring  together  the  best  of  the  hardware  and  publi¬ 
cations  worlds?  ~  ~  ■ 

(a)  Three-Dimensional  (3D)  Simulation  is  the 
first  technique  we  recommend  for  hands-on  troubleshooting 
training,  and  it  should  be  used  wherever  possible. 

Abxandant  data  exist  to  indicate  that,  for  a  large  part  of 
training,  actual  equipment  trainers  (AETs)  can  be  replaced 
with  3D  simulators  that  are  more  training-effective  than 
the  AETs  at  one-third  the  cost  or  less.  Although  AETs  must 
still  be  used  at  the  very  final  segments  of  a  training 
syllabus,  the  3D  hands-on  trouble-shooting  simulator  has  a 
bright  future  when  the  500  million  dollar  annual  purchase  - 


of  Navy  automatic  test  equipment  is  recalled  from  an  earlier 
section  of  this  paper.  Significant  nxambers  of  these  costly 
Test  Stands  are  being  entirely  devoted  to  technician  and 
operator  training.  The  point  is  that  better  training  for 
better  maintenance  for  better  reidifiess  can  be  achieved  at 
less  cost  by  using  simulation  to  reduce  oUr  reliance  on  AETs, 


^  Research  on  such  simulation  is : ongoing  in  a  few  loca- 

throughout  DoD,  but  funding  is  unsystematic  and  spotty. 
The  H^an  Factors  Laboratory  at  NAVTRAEQUIPCEN  has  fortunately 
V  s^PPO^t  of  a  6.3  NAVAIR  program  sponsored 

by  uDR  Paul  R.  Chatelier,  who  is  now  DDR5eE,  Military  Assis¬ 
tant  for  Training  and  Personnel  Technology.  Under  this 
program_j^  our  laboratory  has  developed  a  3D  simulator  for 
the  A- /h  Head-Up  Display  Test  Bench,  as  shown  in  Figure  8. 

Even  including  the  front-end  analysis  of  the  training  re¬ 
quirements  and  performance  specification,  which  is  critical 
in  the  development  of  simulators  like  this,  the  cost  of 
this  device  is  only  approaching  half  that  of  the  actual 
Test  Bench.  That ' s  because  the  simulator  doesn't  have  all 
electronic  wizardry  that  makes  the  actual  equip- 
ment  an  unreliable,  cumbersome,  expensive,  dangerous ,  and 
relatively  training- ineffective  device.  The  Simulator  is 
reliable  and  easy  to  use.  The  trainee  can  walk  around 
the  unit,  pull  out  drawers,  remove  cards,  and  hook  up 
simulated  test  equipment  to  test  points  and  diagnose  a 
malfunction  that  has  been  inserted  by  the  instructor  at 
a  keyboard.  It  permits  hands-on  Intermediate-Level  mainten¬ 
ance  troubleshooting  training  on  the  Head -Up  Display  Set 
and  on  the  Test  Bench  itself  down  to  the  level  of  components 
on  the  printed  circuit  cards. 


There  is  a  6.3  effort  similar  to  ours  being  conducted 
through  Dr.  Marty  Rockway 's  Laboratory  at  Lowry  Air  Force 
Base,  and  we  have  several  experimental  projects  for  applying 
this  technology  through  the  three  levels  of  the  air,  sur¬ 
face,  and  submarine  Navy.  It  can  make  training  at  once 
more  comprehensive,  safer  for  the  trainee  and  the  equip¬ 
ment,  and  less  costly. 


(b)  Automated  JPAs  and  Two-Dimensional  (2D) 
S^P^^lation  are  the  other  elements  that  we  would  include  in 
the  hardware/publications  mix.  It  would  be  technologically 
wasteful  for  today's  solid-state  digital  technology  not  to 
be  applied  to  the  drawbacks  of  the  publications  and  simula¬ 
tion  used  in  isolation  from  each  other.  For  example,  one 
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connnercially  available  device  can  place  3,000  pages  of  text 
(five  average  books)  onto  a  single  4-  by  6 -inch  microfiche. 
Interactive  computer  graphics  could  be  coupled  with  such  a 
device  to  teach  the  novice  technician  either  at  the  training 
site  or  on  the  job. 

While  we  have  accused  paper- covered  JPAs  of  being  out¬ 
moded,  newer  automated  approaches  would  allow  more-respon¬ 
sive  delivery  Systems  to  be  developed  which  could  be  made 
portable  and  usable  on  a  continuous  basis  throughout  formal 
training  and  on  the  job.  For  example ,  a  key  element  of 
a  FOMM  (Functionally  Oriented  Maintenance  Manual)  in  Figure 
7  is  a  Maintenance  Dependency  Chart  which  depicts  every. com¬ 
ponent  in  a  system  in  terms  of  its  dependence  upon  every 
other.  Such  printed  charts  effectively  unburden  the  trouble 
shooter,  but  they  can  now  be  replaced  by  an  interactive, 
microprocess or- driven,  and  highly-portable  device  that  can 
lead  the  technician  step  by  step  in  a  fashion  not  unlike 
the  Fully-Proceduralized  Job -Performance  Aid  of  Figure  7. 

It  would,  in  addition,  present  troubleshooting  sequences  to 
the!  technician  which  are  generative ,  that  is ,  made  up  on 
the  spot  rather  than  pre-printed  and  published  ahead  of 
time  with  all  of  the  concurrent  potential  for  obsolescence. 

Were  such  a  device  packaged  together  with  a  second  unit 
designed  to  train  the  technician  as  well,  we  would  have  a 
phenomenally  useful  tech  data  source  that  is  designed 
specifically  for  '‘traiding’'  the  technician  by  way  Of  a 
single  device.  Research  on  just  such  a  unit  is  ongoing  at 
the  Human  Factors  Laboratory,  NAVTRAEQUIPCEN ,  but  the 
effort  is  in  dire  need  Of  funding  support  at  the  present 
time.  Unfortunately,  no  more  than  a  fraction  of  one  percent 
of  the  annual  DoD  budget  is  being  spent  on  maintenance 
training  research,  a  figure  which  compares  rather  unfavor¬ 
ably  with  the  aforementioned  20%  to  25%  maintenance  cost 
attached  to  the  annual  DoD  budget. 

(2)  Full-Facility  Implementation. 

Now  we  come  to  the  implementation  step  of  our  proposal 
for  a  fresh  approach  to  maintenance  training  and  aiding. 

We  would  like  to  try  the  combination  of  3D  simulation  with 
automated  JPAs  and  2D  simulation  at  selected  DoD  training 
sites.  Once  we  have  successfully  put  together  a  winning 
combination  of  the  presently  known  ‘'^traiding"  media,  we 
should  be  ready  for  an  expeirimental  tryout  at  selected 
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training  sites.  One  lesson  that  we  are  likely  to  learn  at 
the  outset  of  such  a  venture  is  that  we  have  had  an  impos¬ 
sible  pipeline  problem  from  the  very  start.  In  other  words, 
we  may  find  out  that  there  are  reasons  other  than  indadequate 
technician  training  which  account  for  much  of  our  lack  of 
readiness. 

But  supposing  that  we  can  clean  up  the  maintenance 
management  problems,  we  should  pick  training  sites  in  one 
or  more  of  the  services  and  at  least  investigate  the 
possibility  of  simulating  and  "traiding"  the  entire  set  of 
hands-on  courses  taught  at  those  sites.  Both  in  dollars 
and  in  sense  of  security,  the  taxpayer  should  be  happy 
with  the  result  if  we  can: 

(a)  Save  two- thirds  or  mbre  of  the  cost  of  an 
AET  by  simulation, 

(,b)  Supply  the  operator/ technician  of  complex 
equipment  with  training  and  aiding  information  that  will 
assist  him  both  on  and  off  the  field  of  his  operational 
assignment,  and 

(c)  Reduce,  in  the  process,  the  mean-time-to- 
repair  of  a  piece  of  equipment. 

The  overall  point  is  that  modern  maintenance  training 
technology  can  save  training  dollars  while  improving  our 
national  defense  posture. 
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AIRCRAFT  CARRIERS 


FIGURE  2.  COMPARISON  OF  NUMBERS  AND  TONNAGES  OF  THE  VARIOUS  SHIP  CATEGORIES  IN  THE 
U  S.  AND  SOVIET  NAVIES.  (ADAPTED  FROM  NAVSO,  P-3560,  1978) 
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PROCUREMENT, 

MILITARY 

CONSTRUCTION 


FIGURE  3.  SOVIET  RESEARCH  AND  DEVELOPMENT  EXPENDITURES,  EXPRESSED  AS  A 
PERCENTAGE  OF  U.S.  RESEARCH  AND  DEVELOPMENT  EXPENDITURES,  AND 
RESULTING  RELATIVE  SPENDING  FOR  WEAPONS  PROCUREMENT  AND 
MILITARY  CONSTRUCTION.  (ADAPTED  FROM  HEAD,  1978) 


FIGURE  4.  CURRENT  SOVIET  PRODUCTION  OF  SIX  CATEGORIES  OF  MILITARY  EQUIPMENT, 

EXPRESSED  AS  A  PERCENTAGE  OF  CURRENT  U.S.  PRODUCTION  IN  THOSE  CATEGORIES 
(ADAPTED  FROM  HEAD,  1978) 


ACTUALLY  DELIVERED  FOR  SEVERAL  TACTICAL  AIRBORNE  RADAR  SYSTEMS. 
(ADAPTED  FROM  PYATT,  1972) 
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FIGURE  6.  THE  NAVY  SPACE/RESUPPLY  PROBLEM.  UNLIKE  NON-NAVY  COUNTERPARTS,  NAVY 
TECHNICIANS  OFTEN  HAVE  RESPONSIBILITY  FOR  SEVERAL  SYSTEMS,  WITH  VERY 
LIMITED  SPACE  FOR  STORAGE  OF  SPARES  AND  DOCUMENTATION. 
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PERFORMANCE 
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FIGURE  8.  HEAD-UP  DISPLAY  SET  TEST  SET  SIMULATOR  DEVELOPED  THROUGH 
NAVTRAEQUIPCEN  HUMAN  FACTORS  LABORATORY. 
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Prediction  of  System  Performemce  and  Cost  Effectiveness 
Using  Huntcm  Operator  Modeling 

TUastract 

Development  of  modern  air  weapons  systems  must  respond  to  con- 
stemtly  chcmging  threat  conditions.  Continual  increases  in  system 
perfonocmce  must  be  achieved  through  new  designs  or  through  updating 
of  existing  equipment.  Most  approaches  to  system  improvement  involve 
avionics  hardware/software  changes  to  enhance  data  processing  and  dis¬ 
play  or  improve  accuracy  and  quality  of  system  inputs.  As  inter¬ 
dependence  of  system  components  becomes  more  intricate,  the  impact 
of  proposed  changes  can  no  longer  be  directly  estimated  as  em  incre¬ 
ment  to  current  performance.  The  ability  of  the  operator  to  use  new 
capaibilities  must  be  considered  in  the  prediction  of  overall  system 
effectiveness.  Inappropriate  automation  of  system  functions  may 
actually  result  in  performance  decrements  if  not  matched  against 
operator  requirements  throughout  the  mission  timeline.  Sensor  data 
refinements  will  not  change  performeuice  if  they  exceed  the  precision 
which  an  operator  can  use  to  perform  his  functions.  Also  of  import¬ 
ance  in  evaluating  a  proposed  system  is  the  life-cycle  cost  associated 
with  its  implementation.  Alteniatives  may  differ  considercdily  in 
cost  to  accomplish  the  same  objective,  and  would  correspondingly  vary 
in  cost  effectiveness.  Proposed  designs  must  be  examined  both  for 
chctnges  to  current  or  baseline  performance  euid  for  associated 
increases  in  baseline  cost. 

Previous  approaches  to  predicting  system  effectiveness  have 
focused  on  simulation  of  hardware/software  components  without  consider¬ 
ing  an  operator's  ability  or  inability  to  capitalize  on  improved 
features.  The  approach  described  here.  Operator  Interface  Cost 
Effectiveness  Analysis  (OICEA) ,  integrated  major  avionics  system 
Veirieibles  into  a  single  cohesive  model  which  simulates  hardware  and 
softwcire  functions  and  the  performance  of  an  operator  interacting 
with  these  components,  using  a  model  called  the  Human  Operator  Simula¬ 
tor  (HOS) .  Alternative  systems  are  compared  with  respect  to  pre¬ 
dicted  effectiveness  when  used  by  an  operator,  and  costing  techniques 
provide  estimates  of  cost  effectiveness  for  each  approach.  The  tech¬ 
nique  allows  for  system  variation  of  key  factors  impacting  effective¬ 
ness,  including  equipment  2uid  human  reliability,  scenario  features 
and  operator  capabilities.  This  paper  describes  the  OICEA  methodol¬ 
ogy,  docviments  and  applicaction  to  a  fixed-wing  antisubmarine  warfeure 
mission,  and  illustrates  the  Vcuriation  of  systen  and  operator  param¬ 
eters  during  evaluation  of  alternative  designs. 
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Introduction 


Development  and  modification  of  modern  air  weapon  systems  are 
conducted  in  an  environment  of  constantly  changing  and  evolving  threat 
conditions,  in  response  to  a  <dianging  threat  environemnt,  new  systems 
must  be  initiated  and  existing  systems  updated  to  produce  systems  that 
will  be  more  effective  in  countering  both  known  and  presumed  threats. 
With  the  increasing  costs  of  modem  systems  and  the  increasing  heed  to 
demonstrate  return  on  investment,  techniques  have  been  developed  to 
estimate  the  impact  on  systism  performance  of  proposed  hew  designs  and 
updates.  These  predictions  of  future  effectiveness,  usually  obtained 
through  computer  modeling  of  all  or  part  of  the  system,  have  involved 
the  implicit  assumption  that  increasing  system  capabilities  will  yield 
a  direct  and  comparable  increase  in  system  p>erfornance.  However, 
experience  with  the  use  of  new  and  updated  systems  in  the  fleet  has 
indicated  that  this  assumption  is  not  always  tenable.  One  possible 
reason  for  this  is  that  estimates  of  system  performance  from  con¬ 
ventional  hardware  and  software  models  overlook  an  important  system 
component  — —  the  operator.  In  particular,  human  limitations  may 
impose  a  limiting  factor  on  the  utilization  of  system  improvements. 

System  enhancements  usually  involve  either  new  hardware  or  new 
software  that  increase  the  degree  and  quality  of  autcmiation,  or  improve" 
ments  in  the  accuracy  and  sensitivity  of  sensor  information  and  other 

system  input  data.  Conventional  methods  of  performance  prediction 
have  either  ignored  the  operator  or  have  used  simple  transfer  function 
representations,  without  an  adequate  understanding,  of  the  limits  with¬ 
in  idiich  these  functions  are  applicable.  As  systems  have  increased 
in  complexity,  it  is  no  longer  adequate  to  assume  that  automation 
or  computerizing  the  wrong  aspects  of  the  job  may,  in  fact,  have  the 
opposite  effect.  We  can  no  longer  assume  that  by  giving  the  operator 
more  sophisticated  data, systan  performance  will  automatically  be 
improved.  Additional  or  more  refined  sensor  data,  for  example,  will 
not  improve  performance  if  it  is  received  during  periods >of  heavy 
operator  loading  or  if  the  data  exceeds  the  precision  which  the 
operator  needs  to  perform  a  given  function.  Proposed  system  changes 
must  therefore  be  evaluated  by  considering  how  well  an  operator  can 
make  use  of  the  new  capabilities  provided  by  the  propos^  changes. 
Evaluation  techniques  that  do  not  consider  this  factor  in  their 
estimates  will  be  in  error;  as  operator  loading  approaches  saturation, 
such  estimates  will  become  wildly  optimistic. 


Designers  of  Current  and  future  weapon  systems  must  cope  with 
several  harsh  realities  that  constrain  available  design  options.  First, 
funds  for  development  and  deployment  of  systems  are  becoming  limited; 
pressures  are  mounting  for  more  cost-effective  utilization  of  fiscal 
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resources.  Design  decisions  mast  be  mad6  on  the  basis  of  maxi mum 
contribution  to  the  defense  mission  for  each  dollar  invested.  The 
most  effective  design  is  the  one  which  meets  mission  requirements 
at  the  minimum  cost. 

A  second  major  constraint  that  is  being  recognized  more  and  more 
is  that  the  human  is  no  longer  a  low-cost  component  of  systems,  nor 
is  manpower  available  in  unlimited  quantities  with  all  required  skills. 
The  effective  use  of  available  manpower  must  be  a  primary  considera¬ 
tion  in  the  selection  of  design  options;  new  systans  must  meJce  effec¬ 
tive  compromises  between  the  use  of  man  amd  use  of  equipment  to 
accomplish  system  functions,  with  cost-effectiveness  as  a  major 
criterion  for  choosing  between  design  alternatives. 

The  critical  role  of  operator  cape±>ilities  in  limiting  or  enheuic- 
ing  systems  effectiveness,  together  with  the  fiscal  and  manpower  con¬ 
straints  discussed  above,  indicate  the  need  for  a  method  that  will 
predict  with  reasonable  accuracy  what  system  performance  will  be  for 
a  particular  configuration  and  that  will  provide  realistic  estimates 
of  how  that  configuration  will  perform  when  manned  by  operators 
of  V2irying  edsility.  Such  information,  when  ccxnbined  with  appropriate 
cost  data,  would  enable  a  choice  to  be  made  between  alternative  design 
configurations  on  the  basis  of  expected  performance  relative  to 
expected  cost. 

The  work  described  in  this  paper  represents  am  initial  effort  to 
develop  such  cost-effectiveness  prediction  methodology.  The  approach, 
called  Operator  Interface  Cost  Effectiveness  Analysis  (OICEA) ,  uses 
the  Humam  Operator  Simulator  (HOS)  Model  (1,  2),  developed  for  evaluat¬ 
ing  system  operability  during  early  system  design.  The  following 
sections  describe  the  OICEA  methodology,  its  rationale,  and  the  results 
obtained  by  applying  it  to  a  fixed-wing  antisubmarine  warfare  mission. 

APPROACH 

Rationale 

Differing  system  configurations,  proposed  in  response  to  the 
need  for  improved  system  performance,  may  vary  widely  in  effectiveness 
in  their  ultimate  application  but  may  appear  to  be  virtually  indis-  : 
tinguishable  in  potential  value  during  early  development  stages.  Since 
it  is  normally  prohibitively  expensive  to  develop  each  alternative  to 
the  hardware  simulator  or  prototype  stage  in  order  to  obtain  an  esti¬ 
mate  of  its  potential  worth,  there  is  a  need  for  methodologies  that 
Ccm  be  used  ecurly  in  the  system  design  process  to  help  decide  which 
alternatives  should  continue  under  development.  Digital  computer 
modeling  has  been  a  traditional  and  typically  effective  method  for 
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making  such  forecasts  under  such  conditions.  OlCEA  adds  to  the  concept 
of  equipment  modeling  by  combining  it  with  :Jsh6  ability  to  simulate  the 
functioning  of  an  operator  within  the  system  and  by  applying  standard 
costing  techniques  to  derive  projected  life  cycle  costs.  From  these 
data^  alternatives  can  be  compcured  to  one  another  and  to  existing 
systems  capability  on  the  basis  of  cost  effectiveness. 

Any  model  that  is  to  be  used  to  perform  such  an  evaluation  must 
satisfy  several  criteria.  First,  it  must  be  int&gvatvoe.  It  must  be 
able  to  simulate  the  hardware,  software,  and  operator  system  com¬ 
ponents  within  a  single  conceptual  framework,  along  with  any  external 
data  sources  (sensor  data  returns,  communications,  etc.).  Second,  the 
model  must  h&  flexible  t  it  must  be  able  to  accommodate  without  any 
major  revisions  virtually  any  class  of  manned  system  or  subsyston  and 
provide  for  straightforward  modification  of  system  cheuracteristics 
without  extensive  reprogramming.  Third,  it  must  be  sensitive  to 
relatively  subtle  differences  in  configuration  performance.  This  pre¬ 
supposes  a  level  of  detail  in  the  model  consistent  with  the  use  of 
task  or  subtask  level  task-analytic  data  for  operators  and  input/output 
characteristics  for  equipment.  The  sensitivity  criterion  also  demands 
that  the  model  be  dynamic  —  reetLlocating  its  task  priorities  in 
accordance  with  performance  influences  which  may  be  exclusively  time¬ 
line-dependent.  Fourth,  the  model  must  be  peopcmetrie.  Many  of  the 
quemtities  which  describe  operator  capabilities  and  performance 
cluuracteristics  of  equipment  2u:e  not  fixed  values,  but  can  vary  both 
between  eind  within  operators  and  between  and  within  hardware/ sof tweure 
configrurations.  It  may  be  of  great  value  to  iterate  simulations  with 
different  values  of  potentied  key  parameters,  both  to  determine  which 
of  these  particular  paurameters  are  important  in  this  specific  simula¬ 
tion  and  to  identify  parameter  values  that  provide  best  performance. 
Varying  such  quantities  as  software  processing  time  for  a  sensor 
return,  detection  range  or  resolution  for  radair,  amd  operator  recall 
time  for  procedures  can  yield  vatluadsle  information  aibout  design  and 
training  questions  as  well  as  estimating  system  effectiveness.  Fifth, 
the  model  must  be  able  to  produce  specific  qvumtitabive  meastires  of 
system  performance.  There  are,  for  any  system,  numerous  ways  of 
deriving  nianbers  which  reflect  performance.  The  combination  of 
measures  into  a  single  globad  assessment  of  perfonaaince  must  eventu¬ 
ally  entail  obtaining  explicit  judgments  of  worth  or  utility  for  each 
of  these  effectiveness  measures.  The  model,  however,  must  enable 
the  estimation  of  separate  performance  indices  that  are  specifically 
qucmtified>le  and  mission-relevant;  such  measures,  as  time  to  perform  a 
mission,  ordnance  or  stores  expended,  number  of  correct  ship/aircraft 
identifications,  targets  processed  or  probable  kills  are  all  potentiouL 
nimierical  reflections  of  system  success.  While  the  appropriate  meastares 
will  change  from  one  system  to  emother,  effectiveness  indices  should  be 
readily  obtainable  from  routine  model  outputs. 
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The  Human  Opeirator  Simulator 

HOS  is  a  digital  computer  program  designed  to  simulate  the  complex 
interactions  between  man  and  equipment  by  modeling  both  the  operating 
characteristics  of  the  machine  and  the  perceptual,  cognitive  and  motor 
functions  of  the  operator.  HOS  is  a  "generalized  operator."  It 
bto®es  a  operatoJ^  in  a  specific  situation  when  it  is  provided 

with  descriptions  of  equipment  to  be  used  eind  procedures  to  be  followed. 
These  procedural  instructions  are  written  in  a  simplified  English-like 

®®*®pnter  language  called  HOPROC  —  the  Human  Operator  Procedures  liart— 
guage. 


HOS  differs  from  other  models  of  operator  functioning  in  that 
times  for  task  execution  are  not  supplied  by  an  analyst  or  drawn  from 
sample  time  distributions.  Instead,  HOS  generates  task  performance 
data  in  accordance  with  detailed  micromodels  of  human  performance 
built  into  the  HOS  system.  The  HOS  operator  is  capable  of  performing 
seven  primitive  functions"  —  obtaining  information,  remendsering 
information,  performing  mental  calculations,  making  decisions,  moving 
a  body  part,  manipulating  a  control,  and  relaxing.  Every  action  that 
the  HOS  operator  performs  is  a  combination  of  one  or  more  of  these 
primitive  functions.  Internal  decision  rules  within  HOS  will  auto¬ 
matically  determine  the  function  combinations  that  make  up  a  task, 
determine  the  sequence  in  which  tasks  are  performed,  and  Calculate 
the  tMe  required  to  complete  them.  The  HOS  operator  is  goal -oriented; 
that  is,  he  will  perform  actions  necessairy  to  accomplish  a  task,  but 
will  omit  actions  that  have  become  unnecessary  at  some  point  in  time 
due  to  events  elsewhere  in  the  simulation. 

Procedures  and  tasks  to  be  performed  by  the  operator  are  coded 
in  HOPROC  and  broken  down  into  appropriate  micromodel  calls  by  HOS, 
using  a  set  of  internal  algorithms.  Each  of  the  micromodels  assesses 
"time  charges"  against  the  mission  in  accordance  with  its  own  simula¬ 
tion  rules  derived  from  human  performance  data  and  special  experi¬ 
mentation.  Procedures  are  tied  to  one  another  through  a  series  of 
multiplexed  control  routines  and  through  a  "banker"  which  collects 
time  charges  and  records  system  activities  for  later  analysis.  The 
detailed  analysis  of  system  events  is  performed  by  a  routine  called 
HODAC  (Himan  Operator  Data  Analyzer  and  Oollater) ,  which  provides 
information  on  what  the  operator  is  doing  and  what  equipment  is 
involved  at  any  instant  in  time  during  the  simulation.  HODAC  sum¬ 
maries  can  be  used  to  obtain  an  exact  breakdown  of  how  an  operator 
spent  his  time  during  a  mission,  vAiich  sequences  he  executes  and 
how  often,  the  frequency  and  total  time  spent  in  accessing  each  con¬ 
trol  and  display,  along  with  other  summary  reports. 
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HOPRDC,  in  addition  to  being  used  for  coding  operator  procedures, 
is  also  used  for  coding  procedures  which  simulate  the  system  hardware 
and  software  components.  Procedures  or  tasks  to  be  performed  by  equip¬ 
ment  are  coded  in  exactly  the  same  way  as  operator  procedures;  transfer 
functions  or  mathematical  expressions  of  hardware  functioning  may  be 
represented  either  in  HOPROC  or  in  FORTRAN  which  is  a  subset  of  HOPROC. 

HOS  has  several  sophisticated  features  that  make  it  particularly 
applicable  to  the  simulation  of  complex  missions.  For  example,  the 
HOS  operator  has  extensive  decision  making  capabilities  expressed  in 
the  form  of  formal  strategies  or  decision  rules  supplied  by  the  analyst 
that  may  employ  IF  and  branching  logic  eind  that  may  be  dependent  on 
system  status  at  the  time  the  decision  must  be  made.  The  HOS  operator 
has  internal  prioritization  algorithms  that  it  uses  to  determine  what 
to  do  next.  In  making  decisions  about  what  procedure  to  work  on,  HOS 
is  guided  by  two  factors  —  the  original  procedure  priority  (set  by 
the  anod.yst)  and  a  modification  to  that  priority  which  changes  with 

since  that  procedure  was  last  executed.  For  procedures  which 
involve  reading  displays,  control  manipulation,  or  instrument  monitor¬ 
ing,  priorities  are  also  modified  by  a  factor  ccd.led  "internal  l^its," 
which  specifies  the  degree  of  precision  required  for  that  operation. 
Procedures  with  small  internal  limits  must  be  executed  more  often, 
and  priorities  are  changed  accordingly.  Based  on  all  these  factors, 
confuted  priorities  are  campsured  and  the  most  critical  procedure  is 
executed.  The  priority  queuing  model  and  internal  limits  concept  are 
defined  in  detail  in  reference  (3). 

The  ability  to  vary  the  characteristics  of  the  HOS  operator  has 
been  referred  to  earlier,  in  running  the  HOS  model,  a  number  of 
parameters  describing  operator  characteristics  must  be  provided  to 
the  simulation  at  object  time.  The  default  values  of  these  p^^eters 
are  chosen  to  represent  a  trained  operator  of  "average"  capabilities 
who  will  perform  assigned  tasks  with  little  or  no  chance  for  error 
unless  that  error  is  specifically  introduced  and  controlled  by  the 
analyst.  (HOS  will  not  make  procedural  errors  unless  told  to  do  so, 
but  can  forget  information  or  misread  a  number) .  This  power  to  con¬ 
trol  the  characteristics  of  the  operator  provides  a  ready  method  of 
determining  the  range  of  operator  abilities  for  which  a  system  is 
suitable. 

The  HOS  system  has  been  used  to  simulate  a  variety  of  relatively 
simple  tasks  such  as  reach  performance,  multiple  dial  reading  and  mail 
sorting  (4) .  It  has  been  applied  to  assessment  of  operator  workload 
in  a  task  situation  (5)  and  for  simulation  of  a  complex  opera¬ 

tional  mission,  that  of  the  Air  Tactical  Officer  in  the  LAMPS  anti¬ 
submarine  helicopter  (3) .  The  brief  description  of  HOS  in  this  paper 
is  supplanented  by  the  HOS  Study  Guide  (6) .and  by  detailed  micromodel 
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descriptions  in  reference  (3) .  Useful  analyses  which  discuss  HOS  in 
the  context  of  other  operator  models  are  contained  in  Pew,  Baron, 
Feehrer,  emd  Miller  (7) ,  and  in  Greening  (8) . 

Operator  Interface  Cost  Effectiveness  Analysis 

The  approach  to  system  evaluation  described  in  this  paper  is  not 
a  new  model.  It  is  rather  a  way  of  using  a  model  to  organize  cuid 
answer  questions  about  new  systems  or  alternative  designs.  The  goal 
is  to  provide  the  best  possible  projections  of  the  perfoimiance  of 
those  alternatives  under  the  range  of  conditions  likely  to  be  encoun¬ 
tered  in  fleet  use. 

Systems  are  rarely  manned  by  perfect  operators.  While  it  is 
important  to  know  the  performance  potential  of  a  system  if  operators 
were  capable  of  handling  any  task  conceived  by  a  system  designer, 
it  is  much  more  critical  to  understand  the  probable  performance  given 
a  typical  operator  with  human  limitations.  It  is  not  uncommon  for 
substantial  reseaurch  and  development  costs  to  be  invested  in  a  system 
which  performs  a  mission  more  poorly  thcin  a  system  already  available. 
Some  new  systems  achieve  distinct  improvements  in  performance,  but 
only  with  ein  unacceptable  level  of  operating  and  support  costs. 

It  is  never  the  intent  of  designers  to  reduce  performance  or 
increase  costs  beyond  the  acceptable  level.  Frequently,  however, 
these  criteria  are  not  applied  in  choosing  among  the  various  available 
ways  of  obtaining  a  desired  iir^provement  in  system  performance.  The 
premise  of  the  OICEA  approach  is  to  maJce  such  considerations  explicit 
by  the  deliberate  comparison  of  alternatives  to  a  baseline  performance 
level  and  a  baseline  cost.  This  approach,  briefly  stated,  is  the 
systematic  application  of  digitad  simulation  to  derive  cbst/perform- 
ance  data  as  eeurly  as  possible  in  the  design  cycle.  Generally,  this 
will  involve:  (a)  simulating  a  baseline  system;  (b)  simulating 
one  or  more  system  alternatives;  (c)  obtaining  appropriate  perform¬ 
ance  measures  for  baseline  and  adternatives;  (d)  obtaining  baseline 
and  alternative  cost  data;  and  (S)  generating  cost/benefit  tradeoffs 
based  on  these  data.  The  flexibility  offered  by  modeling  allows  per¬ 
formance  estimates  for  a  variety  of  scenarios  and  tactics,  under 
degraded  mode  conditions  and  with  varied  operator  procedures  or 
tacticed  doctrine.  To  achieve  such  a  breadth  of  comparison  using 
prototypes,  dynamic  simulatdrs  or  functioning  mockups  would  require 
excessively  heavy  investments  of  time  and  dollars,  eind  results  would 
be  too  late  to  impact  on  system  selection.  It  is  the  capability  for 
early  identification  of  most  effective  directions  in  development  that 
m5Lkes  the  OICEA  approach  most  promising. 
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OICEA  APPLICATION 

Objective 

The  primary  goal  of  this  initial  application  of  the  OICEA  method¬ 
ology  was  to  determine  if  the  technique,  applied  to  existing  systems 
with  known  problems,  could  identify  those  problems  and  paawide  objec¬ 
tive  data  on  the  type  that  might  have  led  to  correction  had  the  data 
been  available  during  design.  The  Sensor  Station  3  (SS-3)  operator 
workstation  of  the  P-3C  antisubmarine  patrol  aircraft  was  selected 
for  simulation  for  purposes  of  this  demonstration.  The  SS-3  Station 
was  chosen  primarily  because  (a)  three  distinct  configurations  of 
the  SS-3  already  exist  in  the  fleet,  (b)  extensive  documentation  on 
the  system  is  available,  and  (c)  it  is  known  that  this  station  has 
chronic  operator  overload  problmns  during  certain  missions  and  it  was 
desireible  to  conqoare  these  problmns  with  those  identified  from  simula¬ 
tion  output. 

Descriptions  of  the  simulation  that  follow  are  considerably 
abstracted  and  abridged.  Reference  (9)  contains  full  data  on  the 
mission,  scenario  and  operator  tasks,  and  on  specific  equipment 
details  omitted  below. 

Mission  and  Scenario 

The  mission  to  be  flown  was  a  surface  search  of  an  anchorage  area 
off  the  coast  of  a  Mediterranean  third-world  nation.  The  area  df 
interest  was  10x10  nautical  miles  (nm,  j  square.  Primary  objective 
of  the  anchorage  mission  vitas  to  confirm  the  presence  or  absence  of  a 
specific  ship  within  the  area  by  acquiring  Electronics  Support 
Measures  (ESM)  data  from  a  target  matching  the  signatvire  of  at  least 
one  of  the  mnitters  known  to  be  on  that  ship,  followed  by  visual  con¬ 
firmation,  and  the  acquisition  of  Porwaurd-Looking-Infrared  (FLIR) 
pictures  (if  so  equipped)  Of  all  contacts  not  positively  identified 
as  either  neutral  or  friendly.  The  tactical  constraints  were  that 
(a)  flight  within  12  nm  of  the  coastline  was  prohibited;  (b)  total 
^-^lnp  in  the  anchorage  area  vras  to  be  minimized,  (c)  a  single  direct 
overflight  tactic  was  to  be  employed,  (d)  the  aircraft  vroiild  maintain 
2000  feet:  and  180  knots  within  the  anchorage  area,  and  (e)  vessels 
not  in  the  anchorage  eurea  were  to  be  ignored  after  their  location  was 
determined. 

Figvire  1  shows  the  layout  of  the  anchorage  area  with  locations 
of  targets  and  emitters.  There  are  37  total  emitters  of  which  six 
were  targets  of  interest  within  the  anchorage.  Emitters  varied  in  duty 
cycle  amd  in  period  of  emission.  The  aircraft  entered  from  the  initial 
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point  (IP)  to  the  north  and  was  to  fly  from  ta.rget  to  t^get,  in  turn, 
until  all  t^gets  in  the  area  of  interest  had  been  examined. 

Configurations  emd  Tactics 

Four  equipnent  configiirations  of  the  SS-3  station  were  employed: 

(1)  The  Baseline  configuration  was  the  standard  P-3C  without 
FLIR. 

(2)  The  Non-Apriori  configuration  was  the  Baseline  aircraft  with 
the  apriori  filtering  capability  of  the  system  rendered 
inoperative.  The  Apriori  table  performs  preanalysis  of  ESM 
contacts  and  identifies  those  whose  emissions  are  character¬ 
istic  of  particular  target  classes.  This  version  of  the 
simulation  was  run  in  order  to  examine  the  behavior  of  the 
model  in  a  degraded-mode  of  operation. 

(3)  Baseline  +  FLIR.  This  Configuration  was  the  same  as  Baseline 
with  the  addition  of  a  FLIR  system  controlled  by  means  of 

a  joystick  control. 

(4)  Update.  This  configuration  corresponded  to  the  P-3C  Update 
II  with  an  Infrared  Detection  System  (IRDS) ,  essentially 

a  FLIR  system  with  autcxnated  tracking  capability. 

Tactics  varied  among  configurations  as  a  function  of  onboard 
equipment.  Specific  tactics  used  for  each  configuration  were  developed 
with  the  assistance  of,  and  were  approved  by,  fleet  SS-3  operators 
with  recent  experience  in  Mediterranean  anchorage  missions.  The 
tasks  set  for  the  simulated  operator  executing  these  tactics  were 
extremely  complex.  Figure  2  gives  a  general  listing  of  the  classes 
of  these  operator  tasks. 

System  Performance  Measures 


An  cinchorage  mission  is  primarily  an  intelligence  gathering 
exercise,  where  the  information  to  be  obtained  includes  ESM  data  and 
FLIR  pictures,  in  addition  to  the  basic  requirement  of  identifying 
cind  locating  targets  in  the  anchorage  cirea.  Balanced  against  the 
objective  of  maximizing  information  was  the  requirement  that  minimum 
time  be  spent  in  the  anchorage  area.  This  leads  to  two  classes  of 
performance  measures  for  this  mission  —  Amofunt  of  Information  Gathered 
in  this  ease  Emitters  Correctly  Identified  (El)  and  FLIR  Pictures 
obtained  (FP) ,  and  Time  to  Complete  Mission.  Both  of  these  types  of 
measures  cure  dependent  on  the  tactical  situation  and  on  the  locations 


of  targets  and  emitters.  Thus,  the  performance  measures  obtained  for 
a  single  simulation  must  be  considered  as  relative  performance 
indicators;  to  broaden  the  generalizations  from  these  measures,  it  was 
necesszury  to  vary  some  simulation  chauracteristics. 

Sensitivity  Analyses 

Tlxe  discussion  above  pointed  out  that  the  parameters  which  might 
influence  simulation  outcomes  in  a  model  must  be  able  to  be  varied  i 
and  controlled.  The  relative  nature  of  outcomes  noted  in  the  pre¬ 
vious  sections  can  be  overcome  by  systonatically  changing  key  param¬ 
eters  or  mission  characteristics  and  examining  the  robustness  of  the 
findings  to  such  changes.  This  procedure  i^  analogous  to  the  concept 
of  sensitivity  analysis  common  to  model  developments,  in  which  one 
determines  the  range  of  values  for  which  an  equation  or  algorithm 
may  be  Valid,  or  examines  the  sensitivity  of  a  modeled  phenomenon  to 
varidiiions  in  one  or  more  model  parameters. 

This  procedure  of  ea^loring  the  impact  of  changing  characteristics 
on  simulation  outcomes  is  especially  critical  for  diagnosing  the 
reasons  for  differences  in  systems  performance#  If#  for  example/ 
changes  in  operator  capcibillties  produce  little  change  in  the  relative 
rankings  of  alternative  configurations/  it  would  suggest  that  dif-^ 
ferences  between  configurations  are  reflecting  straightforward  eg^uip— 
ment  differences/  unmodified  by  the  limitations  of  the  operator.  On 
the  other  hand,  if  such  operator  variations  should  reverse  the  per- 
fonocuice  rankings  of  configurations,  a  way  of  reducing  operator 
workload  by  redesign  or  by  specific  crew  training  requirements  might 
be  sought. 

As  a  follow-on  to  the  studies  described  in  reference  (9) ,  a 
limited  sensitivity  analysis  was  performed.  Using  the  same  scenario 
and  mission  as  in  the  preceding  study,  the  approach  path  to  ^  the 
anchorage  area  was  changed  from  north  to  west,  display  reading  by  the 
operator  was  degraded  to  introduce  possible  errors  in  display  resolu¬ 
tion,  and  operator  manipulation  time  for  controls  was  increased 
slightly. 

RESULTS  AND  DISCUSSION 

Results  presented  in  this  section  are  primarily  summaries  of  the 
detailed  analyses  performed  on  the  output  of  the  four  simulations. 
In-depth  descriptions  of  results,  including'^  minute-by-minute  activity 
timelines,  specific  aircraft  flight  paths  and  operator  activity 
analyses  by  procedure  are  given  in  reference  (9)#  which  also  contains 
the  complete  HOPROC  coding  and  HGDAC  output. 
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Performance  Measures 


Figi^e  3  shows  a  summary  of  performance  for  each  of  the  four  sys¬ 
tem  conf igilrations .  Time  to  Complete  Mission  varies  dramatically 
across  configurations.  As  the  figure  indicates,  the  Baseline  required 
32  minutes  to  process  and  acquire  89  percent  of  the  Emitters  of 
Ii^terest  (EX) .  The  Baseline  version  wit^  the  apriorl  table  Inactive 
P^O’c®ssed  about  20  percent  fewer  emitters  and  required  one  more  minute 
to  complete  the  mission.  Neither  Of  these  versions  was  ec[uipped  with 
FLIR.  Most  striking  is  the  comparison  of  time  and  effectiveness  for 
the  Baseline,  Baseline  +  FLIR,  and  Update  versions.  When  the  manual 
FLIR  is  added  to  Baseline  in  order  to  permit  FLIR  acquisition  of  data, 
severe  degradation  of  ESM  effectiveness  occurs,  with  a  drop  in  emitter 
acquisition  from  89  percent  to  67  percent,  and  a  minimal  performance 
in  FLIR,  with  less  than  10  percent  (1  of  12)  of  the  possible  FLIR 
pictures  actually  acquired.  Thus,  the  manual  FLIR  addition  not  only 
fails  to  provide  FLIR  capability  as  intended,  but  interferes  strongly 

i-he  ESM  tasks.  This  is  in  distinct  contrast  to  the  Update  version, 
in  vdiich  the  automated  FLIR  improves  performance  on  all  measures, 
accomplishing  100  percent  success  in  both  ESM  processing  and  FLIR 
acquisition,  at  a  savings  of  8  minutes  in  time  over  the  Baseline. 

Analyses  of  operator  activity  show  that  the  ineffective  perform¬ 
ance  of  Baseline  +  FLIR  is  due  primarily  to  the  characteristics  of 
the  FLIR  manual  control.  A  control  slew  rate  of  1.7  deg/sec, too  slow 
for  the  operator  to  overcome  lag  time  in  response  to  aircraft  move¬ 
ment,  was  identified  by  the  HODAC  analysis.  This  problem  could  have 
been  overcome  by  a  straightforward  control  redesign  had  the  deficiency 
been  identified  prior  to  fleet  introduction.  The  capability  to  detect 
and  diagnose  problems  at  this  level  of  detail  from  a  simulated  mission 
indicates  a  distinct  strength  of  the  HOS  and  OICEA  approach. 

Cost 

Cost  data  used  in  this  preliminary  study  reflect  only  Operating 
and  Support  (O&S)  costs,  due  in  large  part  to  the  difficulty  of 
obtaining  accurate  data  on  research  and  development  costs  after  a 
system  is  completed.  For  purposes  of  this  demonstration,  O&S  costs 
are  satisfactory,  although  they  tend  to  be  relatively  insensitive  to 
configuration  differences,  since  the  major. components  of  O&S  costs 
are  only  slightly  affected  by  cheuiges  in  measures  other  than  time. 
Further,  the  O&S  costs  used  here  do  not  reflect  large  differences 
due  to  maintenance  cost  variations  as  would  generally  be  the  case  for 
systems  with  more  variability  in  the  nature  of  equipment  components 
than  those  evaluated  in  this  study. 
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Pigiire  4  shows  the  standaicd  O&S  formula  for  flight  hour  costs 
and  thie  cost/hour  dfetermined  for  the  Beiseline  P-3C.  Figure  5  gives, 
for  each  configuration,  the  on-station  flight  time,  estimated  O&S 
costs  per  flight  hour,  and  the  On-Station  Cost,  a  summ2u:y  value  which 
indicates  tte  total  cost  to  perform  one  mission  from  the  initial  point 
to  area  departure.  The  most  striking  feature  is  the  change  in  mission 
cost  from  $1,961  for  the  Baseline  to  $1,510  for  the  Update,  a  decrease 
of  23  percent  accompanied  by  the  sharp  performance  improvement  already 
described. 

Relative  Cost-Effectiveness 

Two  factors  discussed  earlier  were  the  relative  nature  of  per¬ 
formance  measures  and  the  necessity  for  establishing  a  baseline  of 
current  cost-effectiveness  against  which  proposed  alternative  solu¬ 
tions  could  be  compared.  One  of  the  objectives  of  OICEA  is  to  provide 
guidance  to  designers  and  decisionunakers  about  the  most  fruitful  lines 
of  development  to  solve  a  requirement  for  increased  systSm  performance. 
One  method  of  providing  this  guidance  is  the  concept  of  "acceptance 
regions"  demonstrated  in  Figure  6.  This  figure  displays  data  from 
Figures  4  and  5  in  a  format  which  highlights  the  relative  standings 
of  the  configurations  examined.  In  order  for  a  proposed  solution  to 
be  considered,  it  should  fall  in  or  near  the  acceptance  region,  llie 
size  2uid  location  of  this  region  will  be  governed  by  the  cost-effective^ 
ness  of  current  capability  and  by  other  factors,  such  as  the  importauice 
placed  on  cost  as  zui  evaluation  factor.  Cost  could  be  of  decreased 
weighting  in  the  decision  process  if  the  threat  responded  to  was  suf¬ 
ficiently  critical. 

Figure  6  deals  only  with  performainoe  on  ESM  processing.  A  similar 
figure  could,  of  course,  be  constructed  for  FLIR  performance.  It 
should  be  noted  that  the  use  of  On-Station  Cost  is  a  convenient  way 
of  incorporating  one  performance  measure,  time,  into  the  display  of 
cost-effectiveness  for  another  measure.  The  format  suggested  by  the 
figure  is  only  an  example  of  presenting  cost/performance  information. 

If  research  and  development  costs  had  been  available,  presentation  of 
data  would  have  been  considerably  more  complicated. 

Effects  of  Varying  Utilities 

Simulation  of  the  anchorage  mission  produced  three  measures  of 
system  effectiveness  —  Time,  %EI,  eind  %FP.  These  measxires,  partic-' 
ularly  the  latter  two,  are  mission-specific  and  partly  dependent  on 
the  specific  tattlcal  environment.  In  the  analysis,  they  have  been 
treated  as  separate  indices  of  perfornieuice.  It  would  be  more  desir¬ 
able  when  evaluating  cost/performamce  to  deal  with  a  single  global 
measure  of  perfoimance  which  aggregates  all  possible  measures  of 
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success.  To  do  this,  it  would  be  necessary  to  specify  manerically  vdiat 
each  measure  is  worth  in  the  total  context  of  satisfying  mission 
requirements.  These  indications  of  worth  ax&  the  utilities  of  each 
performance  measure.  Weighting  performance  measures  by  their  judged 
utilities  can  yield  the  desired  global  measure. 

Obtaining  utilities  is  not  simple.  Whereas  a  properly  defined 
mission  requirement  should  identify  what  performance  is  demanded  and 
what  the  associated  utilities  are  as  an  int^ral  part  of  the  require¬ 
ment  statement,  such  data  are  generally  not  paxjvided.  It  may  be 
possible  in  specific  cases  to  obtain  utility  judgments  from  policy 
makers,  but  this  is  not  a  common  practice  at  present. 

Another  way  to  examine  the  effect  of  utility  weighting  is  to 
develop  "boundary  solutions"  to  each  acceptance  region.  This  is  done 
by  systematically  varying  utilities  through  their  probable  ranges  and 
examining  the  changing  locations  of  system  alternatives  relative  to 
the  acceptance  regions.  Although  data  fix>m  this  study  is  not  par¬ 
ticularly  suited  to  such  manipulation  due  to  the  clearcut  superiority 
of  one  version,  an  example  can  be  given  for  the  Baseline  +  PLiR  and 
Update  versions.  If  the  objective  is  to  minimize  On-Station  Cost 
relative  to  total  performance,  and  if  %EI  and  %FP  had  utilities  rang¬ 
ing  from  90/10  (El  is  worth  nine  time  as  much  as  PLIR  performance) , 
to  10/90  (PLIR  nine  times  the  veLlue  of  El),  there  is  combination 
of  utility  weights  which  will  result  in  Update  being  judged  less 
effective.  Thus,  the  Update  configuration  is  superior  regardless 
of  the  "true"  utility  and  the  boundary  solutions  for  acceptance 
regions  are  the  same  for  all  utility  combinations.  It  is  obviously 
easy  to  conceive  of  situations  in  vrtiich  decisions  would  not  be  so 
clearcut  given  a  range  of  utilities.  The  concept  of  determining 
boundaries  for  \idiich  decisions  on  relative  cost-effectiveness  would 
be  unchanged  is  applicable  to  virtually  any  multiple  performance 
measure  problem. 

Effects  of  Varying  Parameters 

As  previously  noted,  several  of  the  characteristics  of  the 
original  simulation  have  been  modified  in  a  separate  series  of  sensi¬ 
tivity  analyses.  These  analyses  sought  to  determine  the  stability 
of  the  initial  simulations  to  changes  in  the  tactical  situation, 
display  resolution,  auid  operator  manual  response  times.  A  secondary 
goal  was  to  explore  the  diagnostic  value  of  these  cheinges  for  pointing 
out  areas  in  which  realistic  design  options  could  be  tested  at  an 
equipment  subcomponent  level.  Each  of  the  three  alterations  defined 
above  will  be  repeated  for  each  of  the  four  SS-3  configurations  and 
results  compared  to  those  from  the  original  simulations.  At  the 


of  this  report,  only  part  of  these  analyses  have  been  completed. 
Pairtlal  results  have  suggfested  that  valuable  information  may  be 

obtained  by  this  process  of  parameter  variation. 

When  a  HOS  simulation  is  run,  a  value  is  input  to  the  model  which 
describes  the  amount  of  time  required  by  am  operator  to  decide  if 
he  can  retrieve  a  given  piece  of  information  from  memory.  This  value 
is  analogous  to  memory  cycle  time  for  a  computer,  and  estimates  the 
time  for  one  iteration  by  the  operator  through  his  "memory  store.” 

This  value  has  been  used  by  HOS  analysts  to  represent  the  degree  Of 
procedural  familiarity  possessed  by  a  simulated  operator.  The  ■'.pre“ 
sumption  of  an  increased  "cycle  time"  is  that  the  operator  with  less 
training  will  be  able  to  rCTiember  procedures  amd  will  execute  them 
properly,  but  will  take  longer  to  recall  the  information  needed  for 
each  procedural  step.  In  the  development  of  the  original  Baseline 
simulation,  this  parameter  was  varied  over  a  range  from  the  default 
value  X.04  seconds)  to  twice  that  number.  The  findings  were  somewhat 
unexpected.  The  time  to  recall  information  has  a  distinct  effect  on 
how  an  operator  allocated  his  time  across  procedures,  but  had  little 
or  no  effect  On  overall  system  performance.  Demands  of  the  system 
were  apparently  such  that  information  recall  time  was  a  minor  element 
compared  to  task  execution  time.  A  tentative  interpretation  of  these 
findings  for  the  Baseline  simulation  is  that ;tn  Baseline  informa¬ 
tion  changed  at  such  a  rate  that  the  need  for  an  operator  to  be  ^le 
to  retain  a  variety  of  types  of  information  "in  his  head"  from  moment 
to  moment  was  not  impoirtant  as  long  as  he  was  adequately  trained  in 
how  to  use  the  system  to  obtain  the  data  when  he  needed  it. 

In  another  variation  on  the  Baseline  simulation,  the  resolution 
available  to  the  operator  from  the  radar  display  was  degraded  to 
reduce  the  positional  accuracy  with  which  a  contact  could  be  located. 
When  the  operator  wishes  to  identify  to  the  system  which  of  several 
contacts  he  wants  to  enter  into  the  onboeird  computer ,  he  manipulates 
a  trackball  which  moves  the  position  of  a  small  circle  (the  hook) 
on  the  display.  When  the  contact  is  encircled  by  the  hook,  it  is 
entered  into  the  onboard  computer  system's  memory  by  a  key  depression. 
The  operator  controls  the  area  covered  by  the  display,  with  radius 
options  of  2  to  1024  mni  in  powers  of  two.  One  change  to  the  Baseline 
configuration  provided  for  a  resolution  uncertainty  of  2  percent  of 
the  distance  between  points  on  the  display.  Given  what  would  appear 
to  be  a  reasonable  error  tolerance,  the  simulated  operator  was  unable 
to  complete  the  HOOK  and  ENTER  TARGET  tasks.  He  spent  large  amounts 
of  time  trying  to  coordinate  the  tracMsall  movements  of  the  hook  with 
the  estimated  location  of  the  contact.  The  2  percent  error  in  resolu¬ 
tion  was  too  great  for  satisfactory  task  performance.  In  a  second 
modification,  resolution  was  changed  to  provide  for  a  tolerance  of 
0.25  percent  of  the  display  radius.  Given  this  error  potential. 
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operator  (and  system)  performance  was  not  detect^ly  different  from 
the  performance  obtained  under  conditions  of  peffect  resolution  in 
the  original  simulation. 

The  examples  above  are  illustrative  of  the  level  of  detail  that 
can  be  achieved  by  controlled  variation  of  Hbs  simulations.  Informa¬ 
tion  such  as  display  resolution  or  sensor  accuracy  required  by  the 
operator  to  do  his  job  could  be  of  considerable  utility  in  determining 
the  equipment  characteristics  of  systems  still  in  the  design  stages. 
Similar  analyses  are  being  conducted  for  variations  on  the  other 
factors  previously  discussed.  Outcomes  of  these  simulations  will  be' 
reported  in  reference  (ID). 

CONCLUSIONS 

Work  described  in  this  paper  was  initiated  in  response  to  the 
observation  that  system  modifications  and  designs,  introduced  to 
improve  total  system  performance,  frequently  had  no  effect  or  an 
adverse  impact  on  that  performance.  The  objective  of  these  initial 
studies  was  to  determine  if  the  OI^!A  approach  could  predict  such  per¬ 
formance  decrements  and  uneconomical  configurations.  This  objective 
has  been  clearly  achieved,  although  substantial  work  is  still  required, 
particularly  in  the  costing  area.  The  Baseline  +  FLIR  simulation 
demonstrates  that  the  addition  of  a  FLIR  sensor  seriously  degraded  ESM 
performance  witii  an  almost  negligible  increment  in  the  amount  of  other 
information  obtained.  The  addition  of  the  FLIR  sensor  was  not  economic' 
®iiy:  justifiable.  The  performance  achieved  by  the  Update  version, 
conversely,  showed  a  substantial  economic  justification. 

These  conclusions  and  the  accuracy  of  the  simulations  themselves 
are  svibstantiated  by  fleet  reports  on  difficulties  experienced  by 
SS-3  operators.  Fleet  operators  are  unable  to  perform  manual  FLIR 
tracking  and  ESM  processing  simultcineously.  One  of  the  other  operators 
must  be  called  on  to  assiuae  one  function  while  the  SS-3  operator  does 
the  other.  This  difficulty  and  its  root  causes  are  clearly  identified 
by  the  simulation.. 

Preliminary  methods  of  analysis  and  display  of  cost-effectiveness 
data  suggest  that  ■the  acceptance  regions  and  boundary  solutions  for 
utilities  derivable  from  model  outputs  can  be  of  great  value  as  design 
and  decision  tools.  Results  of  sensitivity  analyses  suggest  a  con¬ 
siderable  power  for  systematically  exploring  mission  and  operator 
parameters  in  determining  the  robustness  of  simulation  output  to  the 
idiosyncrasies  of  a  single  simulation. 

The  rationale  for  OICEA  is  to  determine  whether  specific  cost  and 
performance  questions  which  should  be  raised  about  all  ndw  designs  and 
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modifications  can  be  answered  through  the  use  of  computer  modeling. 
Our  results,  ,at  least  for  the  mission  and  configurations  considered 
in  these  studies,  clearly  supports  this  approach  to  system  design. 
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Figure  1.  Layout  of  Anchorage  area. 
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Figure  2.  Operator  task  groupings. 
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Figure  3.  Summary  of  system  performance. 
.OSS  COSTS  PER  FLIGHT  HOUR  (FHI-  CR +^RS  +  OOP  + 

WHERE: 

CR  =  COMPONENT  REWORK 

RS  =  REPLENISHMENT  SPARES 

OOC  =  OTHER  OPERATING  CONSUMABLES 

POL  =  PETROLEUM.  OIL,  LUBRICANTS 

SDLM  =  STANDARD  DEPOT  LEVEL  MAINTENANCE 

ER  =  ENGINE  REWORK 

p  =  MILITARY  PERSONNEL 

IOC  -  INDIRECT  OPERATING  COSTS 

•  BASELINE  (P-3C) 

O&SCOSTS/FH  =  2,682  +  1.029  =  $3,711/FH 

Figure  4.  Operations  and  support  (O  &  S)  cost  elements. 
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Figure  5.  On-station  cost  and  data  summary. 


Figure  6.  Relative  costs  effectiveness  analysis 
of  Anchorage  (ESM). 
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Abstract 


The  United  States  Air  Force  School  of  Aerospace  Medicine  and  the 
United  States  Navy  Pacific  Missile  Test  Center  have  developed  a  system 
capable  of  monitoring,  recording  and  analyzing  selected  physiological 
responses  of  atrcrev«iiembers  engaged  in  flying  activities.  The  airborne 
instrumentation  is  battery-operated  and  completely  man-mounted.  Further¬ 
more,  it  is  modular  in  design  to  allow  its  volume  and  weight  to  be  distrib¬ 
uted  over  the  aircrewmember's  torso  in  order  to  minimize  its  interference 
with  his  personal  equipment  and  his  freedom  of  movement.  The  device  can 
record  up  to  20  channels  of  analog  data;  or,  when  combined  with  an  optional 
cardio- thermal  module,  it  can  record  a  mix  of  12  channels  analog  and  32 
channels  digital  data.  The  data  are  multiplexed  on  a  four  track  magnetic 
tape  cassette.  The  specific  data  currently  recorded  include  Inspired  and 
expired  respiratory  gas  flow  rates,  inspired  and  expired  oxygen  partial 
pressures,  ECG/ heart  rate,  skin  temperatures,  acceleration,  cabin  pressure, 
cockpit  voice  communications,  and  a  timing  signal.  The  ground-based  com¬ 
ponents  of  the  system  include  a  data  playback  device,  a  portable  field  data 
processor  for  simple,  straightforward  analysis  in  the  field,  and  a  labora¬ 
tory  data  processor  for  more  rigorous  data  analysis  and  plotting. 


INTRODUCTION 


The  United  States  Air  Force  School  of  Aerospace  Medicine  (USAFSAM) 
and  the  United  States  Navy  Pacific  Missile  Test  Center  (PMTC)  have  been 
engaged  In  a  program  of  Inflight  data  acquisition  and  analysis  for  the 
last  several  years  (Ref.  1-3).  This  program  grew  out  of  a  need  to  mea¬ 
sure,  under  actual  flight  conditions,  the  physiological  responses  of 
pilots  and  other  aircrewmembers  to  the  ever-increasing  physical  demands 
being  made  on  them  by  modern  weapons  systems  and  flying  missions.  The 
objectives  of  this  program  have  been  to  evaluate  the  effectiveness  of 
life  support  equipment  and  systems,  determine  the  oxygen  generation  and 
storage  requirements  for  various  types  of  missions,  accumulate  a  data 
base  from  which  design  criteria  for  new  breathing  systems  and  environ¬ 
mental  control  systems  can  be  developed,  and  assess  the  physiological 
cost  of  flying  operations. 

The  Instrumentation  development  described  herein  began  as  an  attempt 
merely  to  upgrade  the  existing  USAFSAM  physiological  monitoring  device 
(Ref.  4),  but  It  quickly  developed  Into  an  entirely  new  concept.  The  sys¬ 
tem  Is  presently  In  Its  engineering  development  stage,  with  several  re¬ 
finements  yet  to  be  made  before  the  final  designs  are  frozen.  When  fully 
operational,  it  is  expected  to  be  of  significant  value  in  supporting  the 
biotechnology  research  of  USAFSAM  and  PMTC  as  well  as  In  providing  bio¬ 
medical  support  to  test  centers  and  operational  flying  commands,  the  lat¬ 
ter  being  a  rapidly  emerging  thrust  In  the  USAFSAM  program. 


DESCRIPTION 

General 

A  functional  flow  diagram  of  the  total  system  Is  shown  in  Figure  1. 
The  components  of  the  system  are: 

1.  Airborne  Instrumentation 

2.  Data  Reproducer  (playback  device) 

3.  Field  Data  Processor 

4.  Laboratory  Data  Processor  (DECLAB  11/03). 

A  description  of  each  of  those  subsystems  Is  given  In  the  following  para¬ 
graphs. 

Airborne  instrumentation 

Due  to  the  requirements  that  It  be  non-lnvaslve  to  the  aircrewmember 
and  aircraft  and,  when  used  In  fighter-type  aircraft,  not  Interfere  with 


1806 


the  emergency  ejection  sequence,  the  airborne  instrumentation  was  designed 
to  be  completely  man-mounted  and  battery-operated.  The  instrumentation  is 
in  modular  form  so  that  the  various  assemblies  can  be  arranged  in  the  op¬ 
timal  pattern  on  the  aircrewmember's  torso  in  order  to  minimize  its  in¬ 
terference  with  his  personal  equipment  and  his  freedom  of  movement.  That 
feature  also  allows  the  combination  of  modules  having  the  minimum  volume 
and  weight,  conmensurate  with  the  data  requirements  of  each  particular 
study,  to  be  fit  to  the  subject.  The  airborne  instrumentation  is  com¬ 
prised  of  the  data  acquisition  module,  mask  assembly,  flow  transducer 
module,  and  cardio- thermal  module. 

The  data  acquisition  module  is  capable  of  monitoring,  conditioning 
and  recording  data  for  four  continuous  hours.  The  sensors  and  signal  con¬ 
ditioning  consist  of  an  electrocardiogram  (ECG)  amplifier/rate  monitor, 
an  absolute  pressure  transducer,  an  accelerometer,  oxygen  sensor  cir¬ 
cuitry,  and  an  audio  amplifier.  The  conditioned  signals  are  time  multi¬ 
plexed  by  three  8-channel  analog  multiplexers  and  recorded  on  a  standard 
four  track  magnetic  tape  cassette  using  a  pulse  duration  modulation  (PDM) 
format,  as  shown  in  Figure  2.  Each  multiplexer  is  dedicated  to  a  partic¬ 
ular  track  of  the  magnetic  tape.  The  fourth  track  is  used  entirely  for 
audio  recording.  An  internal  clock  generates  a  16-bit  timing  signal  which 
is  initialized  when  power  is  applied.  Four  channels  on  Track  I  are  used 
to  record  timing  information,  leaving  a  total  of  20  analog  channels  for 
data.  The  system  will  accept  any  input  in  the  range  of  0-5  Volts.  Each 
channel  is  sampled  32  times  each  second;  however,  all  multiplexer  inputs 
are  connected  to  an  external  plug  to  facilitate  cross-strapping  of  higher 
frequency  data.  The  six  volt  battery  and  its  associated  ±12  volt  DC-DC 
converter,  also  installed  in  this  module,  power  all  of  the  module's  cir¬ 
cuitry  in  addition  to  that  of  the  oxygen  sensors,  the  flow  transducer 
module,  and  the  cardio- thermal  module.  The  size  of  the  engineering  devel¬ 
opment  version  of  this  module  is  approximately  6.9  x  4.3  x  2.2  inches 
(17.5  X  10.8  X  5.6  cm)  and  weighs  3.3  pounds  (1498  gm).  Further  refine¬ 
ments  are  planned  which  may  reduce  the  size  and  weight  of  this  module. 

The  mask  assembly  is  a  modified  standard  Air  Force  issue  MBU-5/P  oxy¬ 
gen  mask.  The  modifications  are  made  to  facilitate  the  interface  of  two 
sub-assemblies  that  are  used  to  measure  respiratory  gas  flow  rates  and  the 
associated  partial  pressures  of  oxygen.  One  sub-assembly  is  installed  on 
the  inspired  side  of  the  mask,  and  the  other  is  installed  on  the  expired 
side. 


The  inspired  sub-assembly  consists  of  an  aluminum  "T"  fitting,  at¬ 
tached  in-line  to  the  oxygen  inlet  hose,  into  which  the  inspired  oxygen 
partial  pressure  sensor  is  installed.  A  fixed,  sharp-edged  orifice  is 
built  inside  the  fitting,  in  the  path  of  the  inspired  flow.  The  orifice 
is  of  sufficiently  small  diameter  to  produce  a  measurable  pressure  drop 
in  the  line  during  inspiration,  without  causing  excessive  resistance  to 
breathing.  The  pressure  drop  across  the  orifice  is  measured  by  means  of 
pressure  taps  on  the  upstream  and  downstream  sides  of  the  orifice  which 
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connect  via  semi-rigid  small  bore  tubes  to  a  differential  pressure  trans¬ 
ducer  (in  the  flow  transducer  module).  The  differential  pressure  thus 
measured  is  converted  to  a  gas  flow  rate  (liters/min)  measurement  during 
calibration. 

The  expired  sub-assembly  consists  of  a  plastic  fitting  which  clamps 
into  the  expiration  port  of  the  oxygen  mask.  The  expired  gas  is  routed 
through  that  fitting  and  is  then  vented  to  the  atmosphere  through  a  set 
of  three  sharp-edged  orifices,  producing  a  pressure  differential  between 
the  inside  of  the  fitting  and  ambient  conditions.  The  pressure  upstream 
of  the  orifices  is  transmitted  via  a  single  semi -rijgid  small  bore  tube  to 
the  expired  differential  pressure  transducer  (in  the  flow  transducer  mod¬ 
ule).  Only  one  tube  is  required  because  that  differential  pressure  trans¬ 
ducer  is  referenced  to  ambient  conditions.  Additionally,  the  pressure 
created  inside  the  expired  sub-assembly  during  expiration  is  used  to  drive 
a  200  cc/min  (standard  conditions)  sample  of  expired  gas  through  a  drying 
bed  of  molecular  sieve  and  then  to  the  expired  oxygen  partial  pressure 
sensor,  which  is  installed  in  the  fitting.  The  drying  bed  is  necessary 
because  water  vapor  introduces  error  in  the  oxygen  partial  pressure  mea¬ 
surement. 

All  interphone  and  air-to-ground  communications  are  monitored  and 
recorded  by  means  of  a  single  wire  tap  into  the  headset  side  of  the  mask 
interphone  cord.  The  tap  is  made  by  introducing  an  adaptor/connector  be¬ 
tween  the  mask  interphone  cord  and  the  aircraft  interphone  cord. 

The  flow  transducer  module  contains  two  differential  pressure  trans¬ 
ducers  and  their  signal  conditioning  circuitry.  The  transducers  are  con¬ 
nected  to  the  inspired  and  expired  respiratory  gas  collection  sites  by 
means  of  the  semi-rigid  small  bore  tubes  discussed  in  the  previous  para¬ 
graphs.  Power  to  operate  the  transducers  and  their  associated  circuitry 
is  provided  from  the  data  acquisition  module.  The  size  of  the  flow  trans¬ 
ducer  module  is  approximately  1.5  x  2.0  x  3.3  inches  (3.8  x  5.1  x  8.4  cm) 
and  it  weighs  approximately  1.0  pound  (454  gm). 

The  cardio-thermal  module  will  condition,  sample,  digitize,  and  out¬ 
put  eight  body  temperatures  and  ECG/heart  rate.  The  assembly  consists  of 
eight  linear  thermistor  probes,  associated  signal  conditioning,  an  ECG/ 
rate  monitor  and  a  pulse  code  modulation  (PCM)  encoder.  Power  is  provided 
from  the  data  acquisition  module.  A  32-channel  analog  commutator  is  used 
to  sequentially  sample  the  signal  inputs  from  the  thermistor  probes  and 
ECG  electrodes.  The  outputs  of  the  commutator  are  digitized  by  a  PCM  en¬ 
coder.  The  PCM  output  is  a  biphase  1  KHz  signal  which  directly  drives 
Track  III  on  the  recorder  head  of  the  data  acquisition  module.  The  size 
of  this  module  is  approximately  2.25  x  4.0  x  1.0  inches  (5.7  x  10.2  x  2.5 
cm)  and  weighs  0.44  pounds  (200  gm). 

In  the  man-mounted  configuration,  the  data  acquisition  module  and 
flow  transducer  module  are  installed  in  a  modified  SRU-21/P  survival  vest. 
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The  modified  MBU-5/P  oxygen  mask  is  attached  to  the  standard  Air  Force 
helmet  (H6U-26/P),  and  the  cardio- thermal  module  is  worn  under  the  flight 
suit  and  held  in  place  with  a  special  beli  Although  the  airborne  instru¬ 
mentation  will  be  employed  in  the  man-mbtinted  configuration  for  most  ap¬ 
plications,  it  has  been  sized  for  easy  installation  on  the  pilot's  side 
console  of  most  U.S.  military  aircraft.  ■ 

Data  Reproducer 

The  data  reproducer,  or  playback  device,  is  used  to  convert  the  PDM 
signals  recorded  by  the  data  acquisition  module  to  analog  signals.  The 
reproducer  consists  of  a  tape  drive,  a  timing  decoder,  a  binary-to-BCD 
(binary  coded  decimal)  converter,  three  |ignal  integrators,  and  associ¬ 
ated  signal  de-multiplexers.  The  timing  information  is  extracted  from  the 
tape,  converted  to  BCD  and  output  to  a  BCD  display.  The  outputs  of  the 
analog  de-multiplexers  are  connected  to  a  cross-strapping  switch  to  inter¬ 
connect  output  channels  for  higher  frequency  data.  The  voice  channel  is 
fed  into  an  audio  amplifier  and  speaker.  A  24-position  rotary  switch  is 
used  to  sample  any  of  the  analog  outputs  and  display  the  analog  voltage 
on  a  liquid  crystal  display  (LCD)  digital  voltmeter.  When  the  cardio- 
thermal  module  has  been  used  to  record  PCM  data  on  Track  III  of  the  cas¬ 
sette  tape,  those  data  are  reproduced  by  means  of  a  microprocessor-based 
PCM  decommutator.  In  that  case,  cross-strapping  between  channels  is  done 
via  software.  A  keyboard  for  entering  that  information  is  provided.  The 
reproducer  is  capable  of  operating  in  a  pre-programmed  time  mode  in  which 
the  start  and  stop  times  of  the  required  portion  of  data  may  be  entered 
via  the  keyboard  and  an  automatic  time  search  of  the  tape  is  initiated 
and  controlled  by  the  processor.  Outputs  consist  of  30  analog  channels 
and  one  8-bit  digital  channel.  The  reproducer  is  permanently  mounted  in 
an  aluminum  carrying  case  with  a  hinged  lid,  and  is  easily  transportable 
to  and  set  up  at  field  locations. 

Field  Data  Processor 


The  portable  field  data  processor,  designated  yE-80,  was  specifically 
designed  by  PMTC  to  operate  in  a  test  environment.  The  Z-80A  microproces¬ 
sor  was  configured  to  accept  nearly  any  commercial  peripheral  device;  i.e.. 
ZILOG,  INTEL,  ADVANCED  MICRO  DEVICES,  etc.  Additionally,  a  Test  Oriented 
BASIC  language  was  designed  to  simplify  data  acquisition  and  analysis.  The 
field  data  processor  is  primarily  used  for  simple,  straightforward  analysis 
of  data  while  deployed  in  the  field.  The  viE-80  has  integral  analog  multi¬ 
plexers  (32  channels),  a  10-bit  analog-to-digital  converter,  and  digital 
I/O  ports.  The  device  uses  11 K  bytes  of  eraseable-programmable  read-only 
memory  (EPROM)  and  24K  bytes  of  random-access  memory  (RAM). 

The  data  reproducer/field  data  processor  configuration  is  shown  in 
Figure  3.  Analog  data  from  the  data  reproducer  is  transformed  to  digital 
data  by  an  A-to-D  converter  integral  to  the  field  data  processor.  Digital 
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data  from  the  reproducer  (recorded  from  the  cardio- thermal  module)  is 
entered  directly  via  the  digital  input  ports  of  the  field  data  proces¬ 
sor.  Control  signals  from  the  field  data  processor  are  used  to  control 
the  reproducer;  i.e.,  rewind,  fast  forward,  play. 

A  typical  data  reduction/analysis  sequence  is  shown  in  skeletal  form 
in  Figure  4.  The  field  data  processor  sends  out  a  control  signal  to  in¬ 
sure  the  tape  is  rewound  and  then  starts  reading  data.  Data  is  ahalyzed 
in  a  rate-adaptive  manner;  i.e.,  once  initialized,  data  is  ignored  until 
changes  occur  beyond  some  pre-set  threshold  (sensitivity).  Once  the  in¬ 
ternal  buffers  are  filled,  the  reproducer  is  stopped  and  the  analyzed 
data  is  transferred  to  a  digital  cassette  tape  and,  at  the  operator's 
option,  displayed  on  a  printer  and/or  other  terminal  device.  The  process 
then  continues  until  all  the  data  has  been  analyzed. 

The  Test  Oriented  BASIC  interpreter  developed  for  the  vE-80  is  sim¬ 
ilar  to  Dartmouth  BASIC  but  contains  special  features  to  interface  to 
equipment  normally  used  in  a  test  environment.  Also,  all  arithmetic  op¬ 
erations  are  performed  with  an  arithmetic  processing  unit  (AMD  9511)  and 
the  execution  speed  is  therefore  very  fast.  In  a  comparison  (benchmark) 
with  four  other  interpreters  (including  the  DEC  BASIC  for  the  PDP  11/03), 
the  Test  Oriented  BASIC  performed  a  group  of  arithmetic  operations  at 
least  40%  faster  than  any  other  BASIC. 

Laboratory  Data  Processor 

More  rigorous  data  analysis  is  performed  with  the  laboratory  data 
processor,  a  Digital  Equipment  Corporation  DECLAB  11/03  minicomputer. 

Data  analysis  with  that  device  is  similar  to  that  of  the  field  data  proc¬ 
essor  except  that  a  floppy  disk  is  used  for  mass  data  storage  rather 
than  a  digital  cassette.  Additionally,  the  DECLAB  11/03  can  display 
analyzed  data  on  a  graphics  plotter.  It  should  be  noted  that  DECLAB 
analysis  is  performed  in  FORTRAN  and  by  linkages  to  PDP-11  assembly  lan¬ 
guage  routines.  Plans  are  being  made  to  interface  this  machine  to  the 
USAFSAM  computer  network  currently  being  developed.  Once  that  task  is 
accomplished,  it  will  be  possible  to  store  the  accumulated  inflight  data 
in  a  data  base  management  system  for  selective  retrieval  and  manipulation. 

DISCUSSION 


USAFSAM  is  being  tasked  more  and  more  heavily  to  provide  biomedical 
guidance  based  on  inflight  collection  of  physiological  data.  The  present 
data  collection  capabilities  are  presented  in  Table  I.  Planning  is  under 
way  to  expand  the  capabilities  to  include  many  other  types  of  physiolog¬ 
ical  and  environmental  data,  such  as  electromyogram  (EMG),  electroenceph¬ 
alogram  (EEG),  tri-axial  acceleration,  blood  pressure,  expired  carbon 
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dioxide  concentration,  and  blood  oxygen  saturation.  Adequate  sensors 
already  exist  for  some  of  these  measurements,  but,  for  the  others,  suit¬ 
able  non-invasive  sensors  have  yet  to  be  developed. 

The  Inflight  Physiological  Data  Acquisition  and  Analysis  System 
described  in  this  paper  has  been  designed  so  that  new  sensors  and  trans¬ 
ducers  can  be  integrated  into  the  system  with  minimal  effort.  In  addition 
to  having  that  high  degree  of  versatility,  it  is  a  complete  system  which 
takes  data  from  the  collection  site  all  the  way  through  simple  analysis 
and  tabulation  in  the  field  to  more  rigorous  analysis  in  the  laboratory. 
The  system  will  tremendously  enhance  the  ability  to  process  large  amounts 
of  data  very  efficiently  and  provide  rapid  responses  to  test  and  evalua¬ 
tion  requests  from  customers  in  the  field. 
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Audio  Electrical  Tap  to  Intercom  Cord  All  cockpit 

communications 


(displaces  PDM  signals  when 
cardio- thermal  module  is  used) 


Cockpit  j 

Audio 

'  Communications 

Amp 

►Tape  Track  IV 


Figure  2.  Electronic  Configuration  of  Data  Acquisition  Module. 
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Figure  3.  Reproducer/Field  Data  Processor  Configuration 


Figure  4.  Typical  Data  Analysis  Sequence. 
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Synibhetic  Selection  of  Naval  Aviators : 
A  Novel  Approach 

Abstract 


Increased  interest  in  reducing  attrition  in  Under¬ 
graduate  Pilot  Training  has  led  to  the  proposed  addition  of 
a  Synthetic  Selection  System  to  the  traditional  written  ap¬ 
titude  test  battery.  The  system  includes  tests  for  complex 
coordination,  selective  attention  ability,  reaction  to  motion 
stress,  and  selected  cognitive  and  psychomotor  skills.  A 
major  portion  of  the  testing  is  devoted  to  the  last  two 
skills,  using  a  device  embodying  recently  developed  adaptive 
training  and  testing  concepts.  Underlying  use  of  the  device 
is  the  notion  that  learning  will  take  place  in  a  well  de¬ 
signed  testing  environment  with  immediate  feedback.  Hence, 
testing  will  be  structured  to  apply  the  latest  techniques 
for  assessing  and  training  potentially  useful  skills.  The 
device  does  so  by  adapting  a  sequence  of  problems  to  the 
ability  of  individual  students.  Extensive  record  keeping 
is  performed  to  enable  automatic  adjustment  of  adaptation 
rules  to  improve  system  performance.  Data  classifying  stu¬ 
dents  according  to  performance  on  other  tests  and  in  subse¬ 
quent  training  will  allow  the  system  to  cope  with  the 
changing  nature  of  student  input  characteristics  and  down¬ 
stream  training  requirements . 


Introduction 


Recent  history  has  seen  an  acceleration  in  the  cost  of 
Undergraduate  Pilot  Training  (UPT)  within  the  Navy.  This  high 
per  student  cost ^  which  presently  ranges  into  the  hundreds  of 
thousands  of  dollars,  has  been  brought  about  by  the  increased 
cost  of  fuel,  manpower,  and  equipment.  While  fuel  costs  are 
relatively  fixed  in  an  upward  direction,  other  costs  of  Navy 
UPT  could  be  reduced  through  the  increased  precision  of  the 
to61s  used  to  select  the  trainees.  Specifically,  a  reduction 
in  the  drop-out  or  attrition  rate  could  lead  to  a  reduction 
in  the  overall  cost  of  UPT  by  the  concentration  of  time,  ef¬ 
fort,  and  equipment  on  trainees  who  are  more  likely  to  com¬ 
plete  the  program.  In  a  less  economic,  but  equally  important, 
vein  a  more  efficient  selection  program  can  avoid  the  wasted 
time  and  frustration  that  can  occur  when  low  skill/aptitude 
people  are  placed  in  a  program  where  their  chances  of  success 
are  minimal. 

At  the  present  time,  the  aviator  selection  program  con¬ 
sists  of  various  written  tests  designed  to  assess  abilities 
related  to  piloting  skills.  These  tests  are  the  Academic 
Qualification  Test  (AQT) ,  the  Mechanical  Comprehension  Test 
(MCT) ,  the  Spatial  Appreception  Test  (SAT) ,  and  the  Biograph¬ 
ical  Inventory  (BI) .  These  tests  have  been  shown  to  be  valid 
predictors  Of  success  in  UPT.  These  measures,  although  con¬ 
ceptually  sound,  do  not  account  for  a  large  amount  of  the 
variance  in  the  criterion  of  attrition  from  UPT  (North  and 
Griffin,  1977).  The  present  battery  does  a  good  job  of  pre¬ 
dicting  academic  qualification  (only  about  2%  of  failures 
are  academic)  but  does  less  well  in  predicting  motivational 
or  flight  failures  (60.5%  and  21.5%  of  attrition,  respec¬ 
tively,  are  due  to  these  factors).  This  is  not  surprising 
given  the  nature  of  the  criterion  and  the  academic  basing 
of  the  predictors.  Authorities  in  the  area  of  prediction  of 
perfoinnance  in  occupational  settings  have  suggested  the  use 
of  more  behaviorally  based  prediction  measures  (Wernimont 
and  Campbell,  1968).  These  measures,  or  samples  of  behavior, 
are  more  often  likely  to  have  a  stronger  relationship  to  the 
criterion  since  they  contain  more  kinds  of  behaviors  that  are 
also  contained  in  the  criterion.  Various  efforts  have  been 
made  to  improve  the  predictive  efficiency  of  selection  batter¬ 
ies  used  for  UPT  selection  using  behavioral  measures.  Most 
notable  among  these  efforts  is  the  Air  Force's  Automated  Pilot 
Aptitude  Measurement  System  (APAMS)  (Long  and  Varney,  1975). 
This  system  used  a  general  aviation  simulator  (the  GAT-I)  to 
give  brief  flight  instruction  and  automated  testing  to  pro¬ 
spective  aviation  trainees  on  basic  flight  maneuvers.  This 
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me'thod  proved  to  be  a  useful  way  of  predicting  success  in 
UPT  for  Air  Force  student  pilots.  In  keeping  with  the  trend 
toward  more  performance  based  measurement,  the  Naval  Aero¬ 
space  Medical  Research  Laboratory  (NAMRL)  has  stated: 

The  lack  of  any  prominent  breakthrough  in-  per¬ 
ceptual/cognitive  paper-and-pencil  performance 
tests  since  World  War  II  years  suggests  that 
non-paper-and-pencil  performance  tests  should 
be  investigated  to  determine  their  relation¬ 
ship  to  aviator  performance.  (North  and 
Griffin,  1977,  pg  35) . 

In  order  to  follow  through  with  this  mandate,  and  to  improve 
the  predictive  efficiency  of  the  selection  system  thereby 
reducing  attrition  in  UPT,  NAMRL  has  moved  to  add  several 
more  instruments  to  the  selection  battery.  The  following 
describes  these  new  test  devices: 

a.  The  Psychomotor  Test  Device  (PTD)  consists  of  a 
two  axis  tracking  task  that  requires  the  examinees  to  track 
cursors  with  the  simultaneous  coordination  of  a  footpedal 
and  a  joy  stick.  This  is  a  test  of  complex  psychomotor 
coordination. 

b.  The  Dichotic  Listening  Test  (D/L)  requires  exami¬ 
nees  to  respond,  as  directed,  to  either  the  right  or  left 
ear  when  two  messages  are  presented  to  both  ears  simulta¬ 
neously.  This  tests  the  examinees  ability  to  selectively 
attend  to  different  auditory  channels. 

c.  The  Brief  Vestibular  Disorientation  Test  (BVDT) 
determines  the  examinees'  response  to  motion  stress.  They 
are  placed  in  a  rotating  chair  and  required  to  respond  to  a 
digit  repetition  task  while  tilting  their  heads  in  various 
ways . 

d.  The  Integrated  Multitask  Psychomotor  and  Cognitive 
Testing  (IMPACT)  System  is  a  psychomotor  test  device  design¬ 
ed  to  tap  individual  information  handling  and  perceptual- 
motor  capabilities.  In  the  IMPACT  system,  examinees  use  a 
joy  stick  to  keep  a  tracking  cursor  centered  at  a  predesig¬ 
nated  position  on  a  screen  while  carrying  out  the  cognitive 
processing  activity  of  canceling  digits  with  a  key  pad. 
During  the  course  of  this  test  information  is  also  provided 
about  the  degree  of  successful  performance  on  both  tasks 
with  regard  to  an  adaptively  set  goal. 
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In  addition  to  the  above  performance  based  tests,  a 
more  generalized  adaptive  training  and  testing  device  will 
be  evaluated  for  inclusion  in  the  selection  testing  for 
Naval  OPT.  This  device,  based  loosely  upon  the  APAMS  men¬ 
tioned  previously  and  suggested  in  Diehl  (1976),  involves 
the  short  term  training  arid  automated  testing  of  candidates 
for  OPT  on  a  low  cost  aviation  simulator  modeled  after  a 
device  termed  the  trainalator  presently  under  development  at 
the  Htiman  Factors  Laboratory  of  the  Naval  Training  Equipment 
Center.  Each  unit  of  the  system  is  comprised  of  a  Digital 
Equipment  Corporation  PDP-11/34  minicomputer  with  28k  of  MOS 
memory,  a  VT-ll  video  display  unit,  an  RXll  dual  floppy  disk 
unit,  a  Votrax  audio  response  unit,  and  a  stick/ throttle 
quadrant  for  student  inputs.  One  unit  has,  in  addition,  an 
RK-06  cartridge  disk  for  data  base  maintenance. 

The  system  will  be  programmed  to  simulate  the  flight 
dynamics  of  the  T-34C  primary  training  aircraft.  The  graph¬ 
ics  tube  will  display  both  the  instruments  of  the  T-34C  air¬ 
craft  and  the  scene  that  appears  out  of  the  window.  Trainees 
will  use  the  instrments  and  the  visual  reference  to _ accom¬ 
plish  the  maneuvers  during  the  training/testing  sessions. 

The  IMPACT  portion  of  the  selection  battery  will  be  included 
on  the  trainalator  device  and  will  serve  as  an  introduction 
to  the  T-34C  simulation  by  training  examinees  in  the  rela¬ 
tionship  between  stick  movements  and  attitude  indicator  re¬ 
sponses.  During  the  IMPACT  portion  of  the  training/testing 
sessions,  the  examinees  will  be  given  the  same  test  as  de¬ 
scribed  above  under  IMPACT  with  the  exception  that  the 
tracking  task  will  be  done  using  the  attitude  indicator 
rather  than  the  tracking  ball.  This  provides  for  testing  on 
the  IMPACT  system  with  a  high  degree  of  realism  or  face 
validity  and  leads  into  the  early  training  in  instrument-con¬ 
trol  input  relationships  given  on  the  synthetic  selection 
system. 

The  initial  phase  of  synthetic  selection  system  opera¬ 
tion  will  involve  the  instruction  of  the  candidate  in  the 
basics  of  flying  and  will  advance  them  through  the  trainala¬ 
tor  system  to  sophisticated  simulated  flight  maneuvers  such 
as  flying  a  rectangular  course  with  cross  wind.  As  the 
examinees  progress  through  the  syllabus,  the  adaptive  logic 
of  the  program  will  speed  up  the  training  process  by  taking 
individual  trainee  skills  into  account.  In  addition,  the 
computer  system  will  keep  track  of ,  and  store  for  future 
analysis,  trainee  performance  on  the  system.^  The  system  will 
be  comprised  of  modularized  testing  units  which  will  allow 
for  the  expansion  and  addition  of  tasks  as  future^ needs 
dictate.  For  example,  brief  training  in  the  meaning  of  and 


1825 


responses  to  the  annunciator  panel  could  be  given  and  in-  ‘ 
eluded  in  the  training  and  testing  sessions. 

The  flexibility  will  also  provide  for  the  modification 
of  the  system  as  it  responds  to  feedback  from  the  training 
units.  This  will  be  accomplished  by  the  flow  of  data  back 
from  the  various  stages  of  Undergraduate  Pilot  Training. 

This  data  will  be  placed  into  the  computer  system  and  will 
grow  to  form  a  data  base  of  performance  histories.  This 
perfoimance  history  data  will  be  used  for  subsequent  modi¬ 
fication  of  the  selection  system's  cut-off  scores  and  for 
addition  or  deletion  of  test  modules.  As  time  goes  by,  and 
a  large  amount  of  student  performance  data  is  placed  into 
the  system,  a  greater  degree  of  precision  in  predicting  UPT 
success  should  be  attained.  As  the  system  is  refined,  by 
the  flow  of  such  data,  the  prospect  for  using  the  synthetic 
selection  system  in  order  to  determine  to  which  specific 
area  of  flight  training  a  candidate  should  be  assigned  be¬ 
comes  a  further  possibility.  Through  continued  use  the 
system  may  be  able,  based  upon  trainee  scoring,  to  allow  for 
the  determination  of,  not  only  whether  or  not  a  specific 
student  can  be  trained  to  fly,  but  what  type  of  pilot  train¬ 
ing  the  student  has  the  highest  probability  of  completing 
successfully.  Used  in  this  way,  the  system  becomes  a  valu¬ 
able  tool  for  setting  the  maximum  benefit  from  training  by 
making  fine  distinctions  between  students  as  to  their  suit¬ 
ability  for  specific  types  of  training.  In  addition,  as  new 
aircraft  enter  the  inventory  and  training  objectives  change, 
or  the  nature  of  the  ^  student  input  to  the  training  system 
changes,  the  synthetic  selection  system  will  be  automatically 
adapted  or  shifted  in  the  nature  of  the  tasks  taught  and  the 
tests  given. 

Once  installed,  the  synthetic  selection  system  will  be 
subjected  to  an  18-month  long  evaluation  of  the  validity  of 
the  approach.  Five  2 -hour  blocks  per  examinee  are  being  set 
aside  to  run  the  initial  group  through  the  synthetic  selec¬ 
tion  system  during  one  week  testing  periods  prior  to  UPT. 

This  study  will  be  conducted  with  approximately  400  Aviation 
Officer  Cadets  in  the  Navy's  primary  flight  school  at  Pensa¬ 
cola,  Florida.  These  students  will  be  tracked  throughout 
UPT  to  determine  the  degree  of  effectiveness  the  synthetic 
selection  system  training  and  testing  has  on  prediction  of 
attrition  from  UPT.  It  is  expected  that  the  proposed  sys¬ 
tem,  that  features  the  T-34C  simulation  for  training  and 
testing  will  reduce  attrition  substantially,  with  a  possible 
savings  of  $600,000  for  every  percentage  point  of  attrition 
reduced. 
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One  area  in  which  the  synthetic  selection  system 
should  show  the  greatest  promise  is  in  the  reduction  of 
motivational  attrition*  The  selection  process  can  be  con¬ 
ceived  of  as  a  two-way  decision  process.  The  organization 
makes  the  hiring  decision  based  upon  skill  and  adaptability 
inf omation  gathered  from  the  prospective  candidate .  The 
candidate  makes  the  decision  to  join  based  upon  relevant 
information  about  the  organization  arid  the  tasks  he  or  she 
will  be  required  to  perform.  The  synthetic  selection  sys¬ 
tem  provides  a  unique  opportunity  for  prospective  Naval 
aviators  to  gather  accurate  information  about  military  avi¬ 
ation  and  make  a  decision  about  performing  such  an  activity 
on  a  daily  basis.  If  this  decision  can  be  made  prior  to 
participation  in  the  actual  training  program,  a  substantial 
waste  of  time  and  resources  can  be  averted. 

In  that  the  system  represents  an  integrated  testing  of 
specific  skills  directly  related  to  the  activity  of  con¬ 
trolling  an  aircraft  in  flight,  flying  deficiency  attrition 
should  also  be  reduced.  The  short  training  sessions  should 
give  the  Navy  data  about  the  degree  of  training  required  by 
certain  trainees.  Given  the  adaptive  nature  Of  the  system, 
it  can  be  determined  just  how  much  longer  than  average  cer¬ 
tain  trainees  may  require  to  learn  the  flying  skills  in¬ 
volved.  This  allows  for  a  weeding  put,  early  on,  of  the 
trainees  who  might  require  an  excessively  long  training 
period. 

A  further  benefit  of  the  system  lies  in  the  fact  that 
it  is  insensitive  to  irrelevant  characteristics  of  the  exami¬ 
nees.  The  only  aspect  of  the  examinees  that  will  be  moni¬ 
tored  by  the  system  will  be  the  degree  of  progress  through 
the  training  and  the  collection  of  examination  scores.  As  a 
result,  the  system  should  be  less  susceptible  to  charges  of 
unfairness  or  discrimination  based  upon  the  sex  or  race  of 
the  examinee. 

As  data  is  gathered  about  the  progress  of  trainees  in 
UPT  and  fedback  to  the  system,  greater  precision  will  be 
attained  with  regard  to  prediction  to  specific  UPT  pipelines. 
The  future  intention  is  to  channel  the  trainees  into  differ¬ 
ent  avenues  based  upon  their  performance  in  the  synthetic 
selection  system.  Although  this  feedback  aspect  of  synthetic 
selection  is  a  novel  approach  at  the  present  time,  the  near 
term  prospects  for  monetary  savings  and  the  overall  flexi¬ 
bility  and  simplicity  of  the  system  should  make  it  common¬ 
place  in  the  assessment  of  potential  for  success  as  a  pilot. 
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Abstract 


The  problem  of  accurately  predicting  the  overall 
response  characteristics  of  the  Air  Force  Manpower  and 
Personnel  System  CAFMPS]  to  high-level  policy  changes  is 
important  and  difficult.  The  problem  is  important  because 
decision  makers  require  accurate  information  that  is 
plausibly  derived  to  make  good  and  justifiable  decisions. 

The  problem  is  difficult  because  the  AFMPS  is  a  complex, 
functionally  specialized  system  with  response  characteris¬ 
tics  that  are  often  non- intuitive.  Though  each  functional 
specialty  employs  numerous  special  purpose  models  germane 
to  its  own  interest,  no  total  system  model  exists.  Yet, 
much  of  the  important  AFMPS  behavior  results  from  inter¬ 
actions,  interfaces,  and  interdependencies  among  the  func¬ 
tional  specialties.  As  a  result  the  overall  response 
characteristics  of  the  AFMPS  to  policy  changes  is  extremely 
difficult  to  predict  with  any  real  degree  of  confidence. 
Simulating  the  AFMPS  as  a  single,  integrated,  feedback 
control  system  would  provide  decision  makers  with  informa¬ 
tion  of  a  fundamentally  different  type  and  make  possible 
high-confidence  predictions  of  systems  response  to  policy 
changes.  This  paper  describes,  in  three  parts,  the  efforts 
of  the  Air  Force  Human  Resources  Laboratory  to  develop  an 
Integrated  Simulation  Evaluation  Model  (ISEM)  of  the 
AFMPS.  First,  a  conceptual  model  was  devised  to  apply  the 
concept  of  simulation  to  the  AFMPS;  this  is  often  the  most 
difficult  step  in  a  simulation.  Second,  a  small,  scaled- 
down  prototype  was  developed  to  demonstrate  the  feasibility 
of  simulating  the  AFMPS.  The  feasibility  of  this  approach 
was  favorably  evaluated  by  a  panel  of  AFMPS  managers  and 
noted  civilian  scientists.  Third,  the  prototype  is  pres¬ 
ently  employed  in  assessing:  Cl)  model  utility  and  valid¬ 
ity,  (2)  model  sensitivities  to  data  disaggregation  and 
model  detail,  and  (3)  cost/benefit  of  full-scale  development 
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Introduction 


This  paper  describes  the  efforts  of  the  Air  Force 
Human  Resources  Laboratory  to  develop  a  policy  analysis 
tool  capable  of  predicting  and  analyzing  how  the  Air  Force 
manpower- training-personnel  system  responds  to  policy 
changes.  Efforts  have  centered  around  a  model  called  the 
Integrated  Simulation  Evaluation  Model- -^or,  as  it  is 
better  known,  ISEM.  Essentially,  ISEM  is  a  general  pur- 
pose,  large-scale  simulation  of  the  Air  Force  Manpower  and 
Personnel  System  CAFMPS)  and  is  designed  to  facilitate 
policy  analysis.  This  paper  is  a  project  overview  which 
first  discusses  the  problem  of  policy  analysis  in  large, 
complex  systems  like  the  Air  Force  Manpower  and  Personnel 
System  and  the  use  of  simulation  for  coping  with  these 
problems.  Next,  the  model  devised  at  the  Laboratory  to 
apply  the  concept  of  simulation  to  the  problem  of  policy 
analysis  is  presented .  This  will  be  followed  by  a  short 
discussion  of  the  research  problem  which  consists  of  the 
results  of  two  previous  efforts  and  a  problem  evaluation. 
Finally,  ongoing  research  is  discussed. 

Problem  and  Approach 

Every  organization  whether  it  is  large  or  small  has  a 
set  of  rules  and  regulations  designed  to  maximize  organiza¬ 
tional  effectiveness.  When  organizations  are  small,  these 
rules  and  regulations  are  often  quite  easy  to  devise.  If 
two  people  attempt  a  specific  task,  it  may  be  very  evident 
what  policies  are  needed  to  maximize  organizational  effec¬ 
tiveness.  However,  as  organizations  become  large,  complex, 
and  functionally  specialized  like  the  Air  Force,  making 
policy  that  maximizes  organizational  effectiveness  becomes 
much  more  difficult.  For  instance,  policy  made  in  one 
particular  functional  specialty  may  work  extremely  well 
for  that  functional  specialty  but  that  policy  may  have 
unanticipated  consequences  in  other  functional  specialties 
in  that  organization.  These  unanticipated  consequences 
tend  to  be  non- intuitive  and  difficult  to  predict  by  most 
methods.  Thus,  the  problem  of  deciding  what  policy  should 
be  or  what  the  effects  of  a  policy  might  be,  becomes  very 
difficult  in  these  complex  organizations.  However,  the 
concept  of  simulation  seems  to  be  well  suited  for  coping 
with  these  problems.  ISEM  is  a  simulation  of  the  Air 
Force  Manpower  and  Personnel  System--a  model  of  a  system 
that  ’'plays”  upon  a  computer  and  simulates  how  the  system 
behaves  as  time  passes.  Simulation  has  obvious  advantages 
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for  policy  analysis.  To  determine  the  potential  conse¬ 
quences  of  changing  a  particular  policy,  the  policy  change 
is  first  made  within  the  computer,  the  passing  of  time  is 
simulated,  and  the  results  are  analyzed  to  gain  a  better 
idea  of  how  one  might  expect  that  system  to  react  to  a 
particular  policy  change. 

The  project  has  concentrated  upon  providing  the  high- 
level  decision  maker  with  a  policy  analysis  tool  which  is 
fundamentally  different  from  present  policy  analysis 
techniques.  Naturally,  there  are  already  many  models 
within  the  AFMPS.  These  models  tend  to  be  heterogeneous 
and  non- integrated- -and  rightly  so,  since  these  models  are 
designed  for  special  purposes  and  generally  seem  to  perform 
rather  well.  At  best  it  is  difficult  to  perform  high- 
level  policy  analysis  using  a  piecemeal  conglomerate  of 
special  purpose  models.  ISEM  employs  a  "total  systems 
perspective"  and  is  designed  to  analyze  high-level  policy 
issues  such  as:  What  is  a  more  desirable  force  mix  of 
active  and  reserve  forces?  How  might  the  AFMPS  be  expected 
to  behave  if  various  proposed  changes  were  implemented  in 
the  retirement  system?  How  could  force  reductions  be  made 
more  effectively? 

ISEM  does  not  purport  to  solve  any  of  these  problems, 
rather  ISEM  would  provide  the  decision  maker  with  improved 
information.  This  is  important  for  two  reasons.  First, 
in  the  long  run  a  decision  maker's  decisions  Can  be 
expected  to  be  no  better  thanv  the  information  upon  which 
they  are  based.  Second,  after  a  decision  maker  has  made 
the  finest  possible  decision,  it  is  of  little  value  if  he 
cannot  sufficiently  justify  his  decision  to  have  it  imple¬ 
mented.  To  have  a  problem  and  the  concept  of  simulation 
for  coping  with  that  problem  is  one  thing,  but  before  the 
concept  can  be  applied  a  model  of  the  Air  Force  Manpower 
and  Personnel  System  is  needed. 

Model 

The  basis  for  this  model  of  the  Air  Force  Manpower 
and  Personnel  System  was  originated  by  Capt  Jon  Knight  at 
AFHRL  in  1974  and  consists  of  three  elements:  (1)  the 
Internal  Structure  of  the  Air  Force  Manpower  and  Personnel 
System,  (2)  the  environment  that  the  Internal  Structure 
exists  within--the  National  Skills  Market,  and  (3)  a  User 
Interface  System. 
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The  Internal  Structure  of  the  Air  Force  Manpower  and 
Personnel  System  consists  of  three  sub- elements  or  modules. 
This  includes  a  Policy  Information  Control  System,  a 
Training  and  Transportation  Pipeline,  and  a  Personnel 
Force  Structure.  The  Policy  Information  Control  System, 
or  the  PIC,  models  the  network  witich  implements  the  manpower 
and  personnel  policy  within  the  Air  Force. ^  This  is  simply 
to  say  that  it  is  a  policy  data  base.  It  is  the  control 
system  which  controls  the  rest  of  the  model.  Now  consider 
the  Personnel  Force  Structure.  The  Personnel  Force  Struc¬ 
ture  represents  the  skilled  manpower  talents  available  to 
the  Air  Force  with  which  to  accomplish  its  mission.  This 
module  is  essentially  a  personnel  data  base  representing 
aggregations  of  specific  manpower  skills  assigned  to 
particular  units  at  specified  geographic  locations.  In 
the  simplest  terms,  the  model  represents  resources  (the 
Personnel  Force  Structure)  and  the  policies  (the  PIC) 
which  regulate  how  the  resources  are  applied  to  accomplish 
the  mission.  However,  policy  cannot  directly  change  how  a 
resource  is  employed.  Policy  can  specify  what  skill  in 
which  a  resource  should  be  trained  or  where  the  resource 
should  be  located  (assigned),  but  policy  does  not  train  or 
transport.  Rather,  trains,  planes,  and  cars  and  instruc¬ 
tors,  books,  and  schools  do  that--this  function  is  repre¬ 
sented  by  the  Training  and  Transportation  Pipeline.  The 
pipeline  conceptually  connects  the  PIC  with  the  Personnel 
Force  Structure.  These  three  elements  model  the  Internal 
Structure  of  the  Air  Force  Manpower  and  Personnel  System. 

Of  course,  this  exists  within  an  environment  and  that 
environment  is  the  National  Skills  Market. 

The  National  Skills  Market  models  the  influence  that 
the  national  labor  market  exerts  upon  the  Air  Force's 
ability  to  recruit  and  retain  personnel.  When  unemployment 
is  high,  it  appears  to  be  easier  for  the  Air  Force  to 
recruit  personnel  than  when  unemployment  is  low,  though 
our  need  to  recruit  may  be  lowest  when  unemployment  is 
high  due  to  reduced  attrition  and  increased  retention. 

Since  the  ability  of  the  Air  Force  to  recruit  and  retain 
personnel  may  constrain  the  quantity  and  quality  of  person¬ 
nel  available  to  the  mission,  it  is  important  to  consider 
these  factors.  The  National  Skills  Market  module  attempts 
to  model  these  influences. 

The  User  Interface  System  interfaces  the  user  with 
the  simulation  by  providing  information  about  what  has ^ 
occurred  within  the  simulation.  It  will  contain  a  variety 
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of  measures.  On  the  less  exotic  side,  it  will  report 
routine  management  statistics  such  as  PCS  rates,  year-end 
strengths,  and  technical  school  utilization  rates. 

Second,  the  User  Interface  System  will  employ  more  sophis¬ 
ticated  techniques  such  as  goal  programming  to  produce 
complex  measures  and  human  resources  accounting.  One 
example  of  this  type  of  measure  would  be.  What  is  the  most 
demanding  wartime  scenario  that  a  particular  configuration 
of  "faces  and  spaces"  could  meet?  Thus,  the  User  Interface 
System  is  simply  a  way  of  gaining  information  from  the 
simulation. 

This  is  the  basic  concept  that  was  originated  in 
1974.  Naturally  it  has  grown,  it  has  evolved,  it  has 
matured.  Still  recognizable  pieces  of  this  conceptual 
model  can  be  identified  in  the  prototype.  This  description 
greatly  simplifies  ISEM,  but  it  captures  the  essence  of 
ISEM. 

Research  Program 

Based  upon  the  strength  of  this  concept,  the  Air 
Force  Human  Resources  Laboratory  let  a  contract  to  the 
CONSAD  Research  Corporation  in  1975  to  develop  a  methodol¬ 
ogy  for  applying  the  concept  of  simulation  to  the  problems 
of  policy  analysis  in  the  Air  Force  Manpower  and  Personnel 
System.  The  results  of  this  research  appeared  to  be  quite 
promising.  However,  it  is  one  thing  to  have  a  concept  for 
coping  with  a  problem  and  even  a  methodology  for  applying 
the  concept  to  the  problem,  but  it  may  be  much  more  diffi¬ 
cult  to  translate  that  methodology  into  something  as 
concrete  as  lines  of  computer  code. 

Based  upon  the  strength  of  the  concept  and  the  initial 
methodology  work,  the  Air  Force  Office  of  Scientific 
Research  (AFOSR)  became  interested  in  the  research,  and 
along  with  the  Laboratory,  desired  to  demonstrate  the 
feasibility  of  implementing  this  methodology.  Therefore, 
an  ISEM  prototype  contract  was  let.  It  was  funded  by 
AFOSR  and  again  it  went  to  CONSAD  Research  Corporation 
under  the  close  technical  supervision  of  AFHRL.  The 
contract  was  for  a  small,  scaled-down  version  of  ISEM,  but 
the  prototype  is  still  a  reasonable  representation  of  the 
AFMPS.  The  Air  Force  in  the  prototype  contains  91  skills 
(40  officer  skills  and  51  enlisted  skills)  including 
pilots,  navigators,  aircraft  mechanics,  supply  specialists, 
veterinarians,  etc.  It  contains  weapon  systems  like  the 
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B-52,  the  F-111,  and  the  KC- 135.  It  has  17  bases.  These 
include  a  training  base,  a  BMT  base,  and  an  OTS  base.  It 
has  an  APO  on  the  east  coast  and  an  APO  on  the  west  coast, 
two  European  bases,  two  Pacific  bases.,  and  several  CONUS 
bases.  Most  of  the  variables  are  there.  Thus,  the  proto¬ 
type  is  a  small  scaled-down  model  of  the  Air  Force  Manpower 
and  Personnel  System  designed  to  demonstrate  the  feasibility 
of  the  ISEM  concept. 

At  this  point  in  the  research,  the  Air  Force  Human 
Resources  Laboratory  and  the  Air  Force  Office  of  Scientific 
Research  desired  an  independent  objective  evaluation  of 
ISEM  to  determine  the  potential  Value  of  ISEM  to  the  Air 
Force  and  the  appropriateness  of  our  technological  approach. 
To  accomplish  this,  an  evaluation  panel  consisting  of 
military  managers  and  civilians  with  a  rich  mixture  of 
relevant  military  and  civilian  expertise  was  convened  on 
17  and  18  March  1977  in  Washington,  D.C.  As  a  result  of 
the  evaluation  by  this  panel,  it  was  concluded  that  ISEM 
is  a  very  promising  concept  of  potentially  great  value  to 
the  Air  Force.  However,  the  panel,  like  AFHRL,  realized 
that  before  a  wise  and  prudent  decision  could  be  made  on 
the  full-scale  development  of  ISEM,  several  important 
questions  must  be  answered.  They  also  concluded  that  the 
ISEM  prototype  was  quite  powerful  and  should  be  employed 
in  answering  questions  about  model  validity  and  cost/benefit. 
Based  upon  the  strengfh  and  credentials  of  the  persons 
involved  in  the  evaluation  panel,  their  recommendations 
have  served  as  the  basis  for  a  research  plan. 

The  objective  of  the  research  plan  is  to  answer  the 
cost/benefit  questions  associated  with  ISEM  in  order  to 
provide  a  basis  for  a  decision  on  the  full-scale  develop¬ 
ment  of  ISEM.  This  objective  has  been  translated  into 
three  current  research  efforts.  The  first  is  a  Prototype 
Test  and  Evaluation.  The  second  is  a  Sensitivity  Analysis, 
and  the  third  is  the  Cost  Benefit  Analysis. 

The  purpose  of  the  prototype  test  and  evaluation  is 
to  determine  the  potential  utility  and  validity  of  a  full- 
scale  ISEM.  Essentially  the  evaluation  panel  recommended 
two  things.  First,  that  the  prototype  be  installed  on  the 
AFHRL  UNIVAC  1108,  which  has  been  done.  Second,  that 
researchers  work  very  closely  with  potential  users  to  test 
the  capabilities  of  the  prototype  and  to  evaluate  its 
utility  and  validity.  In  an  iterative  cycle,  scenarios 
are  developed  in  close  coordination  with  potential  users. 
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prototype  is  exercised,  and  researchers  interact 
with  potential  users  to  determine  the  reasonableness  of 
the  results.  At  this  point  the  first  step  is  repeated. 
Scenarios  are  refined  and  the  model  adlusted  if  required, 
the  prototype  is  exercised,  and  researchers  again  interact 
with  potential  users.  This  iterative  cycle  continues 
until  the  capabilities  of  the  prototype  for  dealing  with 
that  particular  scenario  are  assessed.  Naturally,  model 
validation  requires  an  in-depth  knowledge  of  how  the  AFMPS 
responds  to  particular  scenarios.  To  devise  these  scenarios 
a  working  group  of  potential  users  has. been  formed.  Publi¬ 
cation  of  these  results  is  scheduled  for  FY80. 


j  utility  and  validity  of  the  ISEM  prototype 

and  the  ISEM  concept  are  assessed,  the  next  questions  that 
occur  are:  What  are  the  appropriate  levels  of  data  aggre¬ 
gation?  What  IS  the  appropriate  level  of  model  detail? 
wnat  types  of  operating  characteristics  are  required  to 
provide  meaningful  answers  to  real  world  Air  Force  manpower 

will  be  answered  by  the 

sensitivity  analysis  which  is  investigating  the 
effects  of  varying  levels  of  model  detail  and  aggregation. 
This  consists  of  essentially  two  steps.  First,  the  range 
of  scenarios  which  ISEM  should  address  will  be  assessed, 
and  based  upon  that  assessment,  the  variables  will  be 
manipulated  and  the  interactions  analyzed.  Results  will 
be  published  in  FY80. 


With  the  results  of  the  ISEM  Prototype  Test  and 
Evaluation  research  and  the  ISEM  Sensitivity  Analysis,  it 
then  becomes  feasible  to  determine  what  the  cost  and  bene - 
tits  of  an  ISEM  might  be.  This  is  the  thrust  of  the  third 
research  effort.  The  ISEM  Cost/Benefit  Analysis  will 
determine  the  cost  and  benefits  of  development.  This  will 
e  a  difficult  task.  First,  review  the  methodologies  and 
techniques  for  assessing  th^  cost  and  benefit  of  large- 
scale  simulations  such  as  ISEM  which  produce  non-market 
valu®d  information.  Then,  perform  the  analysis.  Results 
of  research  should  also  be  published  in  FY80.  Results 
from  these  research  efforts  will  provide  the  basis  for 
determining  the  feasibility  of  development  of  ISEM. 

As  a  part  of  ISEM,  AFHRL  is  also  pursuing  the  develop¬ 
ment  of  a  National  Skills  Market  model.  This  research  is 
being  pursued  somewhat  independently  since  research  in  the 
National  Skills  Market  has  implications  for  many  other 
potential  users  besides  ISEM.  Because  of  this,  the  National 


Skills  Market  research  and  the  ISEM  research  are  concep¬ 
tually  separated  in  the  AFHRL  research  program.  However, 
all  research  in  the  National  Skills  Market  is  being  coordi¬ 
nated  to  insure  that  the  resulting  product  can  support 
ISEM  requirements  for  labor  market  data.  The  purpose  of 
the  National  Skills  Market  model  is  to  determine  how  the 
labor  market  affects  the  ability  of  the  Air  Force  to 
recruit  and  retain  required  personnel.  Presently  the 
National  Skills  Market  has  been  broken  into  two  sections. 
First,  research  into  the  internal- external  labor  market 
interface  will  attempt  to  determine  how  people  within  the 
Air  Force  and  potential  recruits  make  their  decisions  to 
enter  or  leave  the  service.  Then  in  the  National  Skills  Market 
submodel,  research  will  attempt  to  determine  in  which 
submarkets  of  the  national  labor  market  the  Air  Force 
competes,  who  the  other  competitors  are,  and  what  the 
market  structure  is. 

In  summary,  ISEM  offers  substantial  promise  for  pro¬ 
viding  Air  Force  decision  makers  with  a  new  policy  analysis 
tool  of  a  fundamentally  different  type.  While  there  are 
several  important  questions  about  ISEM  that  remain  unan¬ 
swered,  research  is  underway  to  answer  these  questions  so 
that  a  wise  and  prudent  decision  can  be  made  on  the  full- 
scale  development  of  ISEM. 
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EVOKED  BRAIN  POTENTIALS  AS  PREDICTORS  OP  PERFORMANCE: 
HEMISPHERIC  ASYMMETRY  AS  RELATED  TO 
PILOT  AND  RADAR  INTERCEPT  OFFICER  PERFORMANCE 

Abstract 


Training  a  single  Navy  pilot  to  combat  readiness  is 
estimated  to  cost  about  $500,000.  Every  year  several  air¬ 
craft  ,  each  costing  millions  of  dollars,  are  lost  in  flight 
accidents.  Improved  pilot  selection  would  reduce  these 
costs. 

A  great  deal  of  effort  has  been  expended  since  World 
War  i  in  the  attempt  to  select,  from  the  pool  of  applicants 
to  flight  training,  those  with  the  greatest  potential  for 
successful  completion  of  training  ahd  for  high  level  post- 
tralriing  performance.  Despite  intensive  effort  using  paper 
and  pencil  tests,  psychomotor  tests,  and  other  approaches, 
attrition  in  pilot  training  has  averag^ed  around  30/5  for 
the  past  several  decades. 

The  present  research  was  an  exploratory  study  intended 
to  determine  the  utility  of  a  newly  emerging  technology— 
computer-averaged  brain  wave  analysis— as  a  means  of  improv¬ 
ing  the  selection  of  naval  aviators.  During  the  past  de¬ 
cade,  research  in  several  laboratories  has  revealed  sur¬ 
prising  differences  in  the  functions  served  by  the  left 
hemisphere  (LH)  and  right  hemisphere  (RH)  of  the  human 
brain.  These  studies  used  as  subjects  patients  in  whom  it 
had  been  medically  necessary  to  sever  the  bridge  of  nerve 
fibers  linking  the  LH  to  the  RH.  The  LH  serves  functions 
characterized  as  verbal,  logical,  and  sequential.  The  RH 
processes  information  of  a  different  kind,  and  in  a  different 
way — functions  characterized  as  three-dimensional,  simulta¬ 
neous,  Judgmental,  and  intuitive.  Our  hypothesis  was  that 
these  difflcult-to-measure  functions  of  the  right  hemisphere 
might  be  those  especially  important  in  aviator  performance, 
and  might  be  measured  through  the  use  of  the  hew  computer- 
averaged  evoked  potential  technique. 

Subjects  were  28  Navy  pilots  and  30  radar  Intercept 
officers  (RIOs)  who  volunteered  to  be  tested.  Eight  channels 
of  evoked  potential  data  were  gathered  from  scalp-contact 
electrodes.  Statistical  analysis  showed  consistent  differences 
in  brain  wave  measurements  between  pilots  and  RIOs,  and  within 
pilot  and  RIO  groups,  between  those  rated  as  high  perfor¬ 
mers  vs  those  rated  as  low  performers  by  their  Superior. 


EVOKED  BRAIN  POTENTIALS  AS  PREDICTORS  OP  PERFORMANCE : 

HEMISPHERIC  ASYMMETRY  AS  RELATED  TO 
PILOT  AND  RADAR  INTERCEPT  OFFICER  PERFORMANCE 

Introduction 

The  personnel  costs  in  naval  aviation  are  extremely  high. 
Training  a  single  Navy  pilot  to  the  point  of  combat  readiness 
is  estimated  to  cost  nearly  $500,000.  Attrition  of  pilots 
in  training  has  averaged  30  percent  during  the  past  several 
decades.  Even  pilots  who  wash  out  early  may  represent  k 
loss  of  several  hundred  thousand  dollars.  Further,  human 
error  is  a  significant  cause  of  aircraft  accidents,  and  re¬ 
sults  in  the  loss  of  a  number  of  multi-million  dollar  aircraft 
each  year.  Improved  aviator  selection  could  reduce  these  costs. 

Intensive  attempts  to  devise  methods  for  selecting,  from 
the  pool  of  applicants  for  flight  training,  those  with  the 
greatest  potential  for  effective  performance,  first  as  train¬ 
ees  and  later  as  pilots  and  other  flying  officers,  have  been 
carried  out  by  the  various  military  services  for  the  past  half 
century.  By  and  large,  these  efforts  have  been  reasonably 
successful — as  successful  as  available  technology  would  permit. 
A  wide  variety  of  paper  and  penc-ll  tests,  psychomotor  tests, 
neurological  tests,  etc.  have  been  experimentally  evaluated. 
Despite  rigorous  efforts  to  improve  aviator  selection,  it  has 
not  proven  possible  to  reduce  the  attrition  rate  among  trainees 
from  its  present  level  of  approximately  30  percent. 

Aviators  represent  a  highly  selected,  highly  elite 
group.  Nevertheless,  one  can  assert  with  confidence  that 
there  remains  a  wide  range  of  ability  not  only  among  those 
who  are  selected  for  aviation  training,  but  even  among 
those  who  survive  the  training  and  become  full-fledged  combat 
aviators. 

Most  of  us  vastly  underestimate  the  range  of  human 
abilities.  Whenever  it  has  been  possible  to ^measure  any 
dimension  in  which  humans  vary--whether  that  dimension  has 
been  physical,  chemical  or  behavioral — the  range  of  varia¬ 
tion  has  been  truly  enormous.  For  example,  in  some  of  the 
studies  conducted  by  Industrial  psychologists,  production 
supervisors  have  been  asked  to  estimate  the  range  of  differ¬ 
ences  in  performance  between  the  most  and  least  productive 
of  their  experienced  workers.  Their  estimates  usually 
range  from  about  10-30  percent,  that  is,  they  say  the 
better  workers  are  perhaps  30  percent  more  effective  than 
the  least  productive  employees.  Yet,  when  actual  "hard" 
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production  figures  are  gathered,  pven  for  such  simple  tasks 
as  typing,  card  punching  and  machine  sewing,  results  show  that 
the  productivity  differences  usually  range  from  200-300 
percent.  As  the  difficulty  and  complexity  of  the  task  in¬ 
crease,  the  range  increases.  For  example,  a  study  of  the 
time  taken  by  computer  programmers  to  complete  a  standard 
program  showed  a  range  difference  of  2300  percent.  No 
doubt  "hard"  measures  of  military  combat  pilot  performance 
would  yield  equally  large  differences  in  performance. 

How  can  we  measure  or  predict  these  individual  differ¬ 
ences  in  such  a  way  as  to  permit  us  to  select  from  a  group 
of  applicants  those  most  likely  to  turn  out  to  be  the  best 
performers  ? 

A  recent  review  titled  "Aviator  Selection  1919-1977*'* 
published  by  the  Naval  Aerospace  Medical  Research  Laboratory 
lists  1^15  items  in  its  bibliography,  describing  a  truly 
enormous  range  of  bright  ideas  that  psychologists  and 
others  have  generated  during  a  half  century  of  concentration 
on  the  problem.  The  authors  of  the  Aerospace  Medical  Lab 
report  observe  that  despite  all  the  research,  "current 
selection  tests  normally  account  for  less  than  half  of  the 
total  variance  associated  with  aviator  Success  in  training," 
They  also  comment,  "...lack  of  any  prominent  breakthrough... 
since  the  war  years  (WW-II)  suggests  that  non-paper  and  pen¬ 
cil  performance  tests  should  be  Investigated  more  fully..." 

Psychobiological  Approach  to  Selection 

The  past  ten  years  have  produced  some  extremely  inter¬ 
esting  developments  in  the  field  of  psychoblology“develop- 
ments  that  may  lead  to  major  improvements  in  personnel  selec¬ 
tion  technology. 

The  first  of  these  developments  concerns  the  availabil¬ 
ity  of  new,  highly  sophisticated  electronic  devices  for  re¬ 
cording,  amplifying  and  analyzing  complex  bloelectrlcal  sig¬ 
nals,  such  as  those  which  emanate  from  the  human  brain.  The 
evoked  potential  technique  is  now  being  investigated  in  many 
laboratories  around  the  world.  Evoked  potentials  (EPs)  are 
minute  electrical  brain  waves  which  are  produced  by  sensory 
stimulation.  They  are  ordinarily  obscured  by  larger  ampli¬ 
tude  ongoing  electroencephalographlc  (EEG)  activity.  Advan¬ 
ces  in  electronics  and  computer  design  permit  the  recording 
and  measurement  of  EPs  with  a  high  degree  of  accuracy  and 
reliability.  The  use  of  the  computer  to  record  and  average 
the  EP  so  that  it  may  be  seen  against  the  background  noise 


of  the  EEG  has  provided  a  dramatic  upsurge  of  interest  in 
the  field  of  psychobiology. 

The  other  major  development  in  the  field  of  psychobio¬ 
logy  stems  from  our  newly  emerging  understanding  of  the 
different  functions  of  the  right  and  left  hemispheres  of 
the  brain.  These  findings  are  ah  outgrowth  of  the  attempt 
to  treat  severe  incapacitating  epilepsy  by  surgically  sever¬ 
ing  the  corpus  callosum — the  bridge  of  nerve  fibers  that 
connects  the  two  hemispheres.  Research  on  patients  on  whom 
such  surgery  was  necessary  has  revealed  surprising  differ¬ 
ences  between  the  information  processing  functions  of  the 
two  halves  of  the  brain.  Briefly,  the  function  Of  the 
dominant  hemisphere  (the  left  for  most  people)  is  to  pro¬ 
cess  verbal,  logical, "rational”  information  in  a  sequential, 
linear  fashion.  The  function  of  the  other  hemisphere  (the 
right  hemisphere  in  most  people),  which  has  been  described 
as  spatial,  non-linear,  simultaneous,  judgmental,  holistic 
and  intuitive,  is  not  as  yet  well-defined.  Actually,  of 
course,  virtually  all  tasks  require  the  use  of  both  hemis¬ 
pheres.  However,  when  you  look  at  a  photograph  of  a  crowd 
and  try  to  pick  out  the  faces  of  the  people  you  know,  your 
judgmental  right  hemisphere  is  working  harder.  When  you 
compose  a  letter,  solve  an  equation,  or  answer  a  multiple 
choice  question,  your  logical  left  brain  bears  most  of  the 
load. 

An  analysis  of  Einstein's  writings  has  led  some  re¬ 
searchers  to  conclude  he  solved  problems  by  a  brilliant 
creative  visualization  of  the  solution  (probably  right 
hemisphere),  then  followed  with  a  meticulous,  painstaking 
(left  hemisphere)  mathematical  proof. 

Actually,  the  discovery  of  the  differing  functions  of 
the  right/left  hemisphere  of  the  brain  has  been  anticipated 
for  a  long  time,  primarily  by  writers  and  philosophers. 
Figure  1,  taken  from  Robert  Ornsteln's  book.  The  Psychol¬ 
ogy  of  Consciousness ,  shows  the  variety  of  proposals  that 
have  been  made  by  various  writers  which  contrast  what  Orn- 
stein  refers  to  as  "the  two  modes  of  consciousness."  The 
middle  column  shows  the  many  terms  which  we  may  use  in 
helping  to  describe  left  hemisphere  function,  while  the 
right  column  helps  better  understand  the  kinds  of  concepts 
which  have  been  applied  to  right  hemisphere  function. 

Figure  2,  also  taken  from  Ornsteln's  book,  shows  the 
responses  from  a  split-brain  patient  to  the  request  that 
he  write  the  word  "Sunday"  and  copy  two  figures;  St  cross. 


THE  TWO  MODES  OF  CONSCIOUSNESS:  A  TENTATIVE  DICHOTOMY 
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Figure  1.  Concepts  relating  to  hemispheric  function  (per  Ornsteln) . 


and  a  cube.  With  his  right  hand,  which  is  primarily  driven 
by  the  left  hemisphere,  the  subject  was  able  to  process  ver¬ 
bal  material.  As  you  can  see,  he  can  write  the  word  "Sun¬ 
day.”  However,  as  you  can  also  see,  he  was  totally  unable 
to  copy  the  cross  or  the  cube.  With  his  left  hand,  how-^ 
ever,  he  was  unable  to  cope  with  the  verbal  demand  to  write 
the  word  "Sunday,"  but  he  did  a  recognizable  job  of  copying 
the  cross  and  the  cube. 

How. do  these  findings  from  the  field  of  psychobiology 
relate  to  the  problem  of  aviator  selection,  or  for  that 
matter,  the  selection  of  personnel  for  any  type  of  train¬ 
ing  or  duty?  Most  testing  for  personnel  screening  and 
assignment  to  training,  whether  in  the  military  or  civilian 
sectors,  is  based  on  paper  and  pencil  tests.  Such  tests 
do  a  fair  to  good  job  of  identifying  those  who  will  succeed 
in  school-type  settings,  such  as  civilian  schools  and 
colleges,  and  military  training.  Unfortunately,  however, 
paper  and  pencil  tests  leave  much  to  be  desired  in  predict¬ 
ing  successful  on- job  performance.  Psychologists  have  been 
butting  their  heads  against  this  stone  wall  for  well  over 
half  a  century. 

In  my  first  20  years  of  doing  personnel  measurement 
research  for  the  Navy,  I  constructed  and  evaluated  perhaps 
50  different  paper  and  pencil  tests,  most  of  these  speci¬ 
fically  designed  to  predict  success  in  real-life,  as  opposed 
to  academic,  settings.  After  the  50th  failure,  I  thought 
it  might  be  time  to  try  a  new  tack. 

A  few  years  ago  I  concluded,  from  avocatlonal  reading 
in  the  field  of  psychobiology,  that  despite  the  efforts  by 
myself  and  others  to  predict  performance  in  non-classroom 
situations,  the  tests  we  were  developing  were  primarily 
aimed  at  what  was  beginning  to  be  called  left  hemisphere 
function.  It  seemed  however,  that  perhaps  we  should  be 
aiming  at  the  other  half  of  the  brain — an  elusive  target, 
gvoked  potential  technology.  It  seemed  to  me,  might  be  a 
way  of  getting  at  the  prediction  of  the  hard-to-characterize, 
practical,  real-life  behaviors  which  determine  how  well 
people  do  in  real  jobs,  such  as  piloting  airplanes  or  operat¬ 
ing  sonar  gear.  With  these  ideas  in  mind  (both  hemispheres), 
we  embarked  on  a  program  of  exploratory  research.  We  have 
completed  several  studies  so  far,  including  one  on  reading- 
disabled  recruits,  and  another  on  sonar  operator  performance. 
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Homl spheric  Asymmetry  in  Aviators 

The  present  study  was  intended  to  determine  whetlu-vr 
p5ychobloloj;^lcal ,  specifically  evoked  f)<jtt*nt  la  I ,  te'ChnoLogy 
might  ijrovlde  Information  about  individual  differences  in 
brain  function  which  would  allow  us  to  make  predictions  about 
the  performance  of  naval  aviators.  We  were  particularly  Inter¬ 
ested  in  the  possibility  that  those  men  showing  larger  brain 
wave  amplitudes  In  their  right  hemispheres  than  in  their  left 
might  prove  to  be  better  prospects  for  being  successful  pilots 
than  men  showing  the  opposite  pattern.  These  differences  in 
amplitude  between  the  right  and  left  hemisphere  are  referred  to 
as  "hemispheric  asymmetry."  In  the  case  of  the  pilot,  there 
is  a  great  demand  that  he  be  able  to  operate  effectively  in 
three-dimensional  space,  and  to  make  split-second  judgments  by 
weighing  a  number  of  disparate  variables  essentially  simul¬ 
taneously.  These  demands  seem  to  place  a  great  burden  on  the 
right  hemisphere. 

We  also  hypothesized  that  radar  intercept  officers  (RIOs) 
might  be  found  to  show  the  opposite  configuration  of  hemispheric 
amplitude.  RIOs,  while  they  certainly  must  have  a  good  deal  of 
three-dimensional  imagination  and  must  think  quickly,  are  re¬ 
quired  to  perform  operations  in  a  logical,  sequential,  orderly 
manner  to  function  effectively. 

Obviously,  pilots  must  have  both  hemispheres  functioning 
at  a  high  level  to  be  effective  pilots,  and  the  same  is  true 
of  RIOs.  Our  hypothesis,  then,  was  that  while  both  hemispheres 
must  be  functioning  effectively  in  both  pilots  and  RIOs  (inso¬ 
far  as  our  instrumentation  would  permit  us  to  make  such  Judg¬ 
ments),  pilots  would  show  greater  asymmetry  in  favor  of  the 
right  hemisphere,  while  RIOs  would  show  relatively  greater 
asymmetry  favoring  the  left  hemisphere.  We  assumed  that  these 
differences,  if  found,  would  be  the  result  of  the  selection  and 
attrition  pressures,  self-initiated  and  imposed  from  without, 
which  impinge  differentially  upon  pilots  and  RIOs. 

We  also  generated  a  hypothesis  regarding  aviator  proficien¬ 
cy.  We  predicted  that  within  the  pilot  group,  the  pilots  who 
were  regarded  as  being  superior  performers  would  show  more 
right  minus  left  asymmetry  than  the  lower  rated  flyers,  whereas 
the  opposite  would  be  true  among  RIOs. 

Data  Collection 

The  subjects  in  our  study  were  28  pilots  and  30  RIOs 
Mssigned  to  a  Readiness  Training  Squadron  at  the  Miramar  Naval 
Air  Station  in  San  Diego.  Approximately  half  of  each  group 
were  instructors  and  the  other  half  students.  We  explained 
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the  purpose  of  the  study  to  the  subjects  and  obtained  their 
permission  to  test  them.  Althoup.h  participation  was  volun¬ 
tary,  all  the  subjects  were  Interested  In  tlie  experiment  and 
cooperated  willingly* 

The  testing  took  place  in  the  mobile  NPRDC  experimental 
van,  which  we  were  permitted  to  park  in  the  squadron  hangar. 
Figure  3  shows  the  van  parked,  ready  for  the  testing  to  begin. 

Figure  4  shows  our  testing  setup.  Note  the  cloth  helmet 
with  the  eight  electrode  tubes  attached,  which  was  worn  by  each 
of  the  subjects  during  testing.  Sponge-tipped  electrodes, 
moistened  with  an  electrolytic  solution,  were  placed  in  each  of 
the  electrode  tubes,  in  contact  with  the  subject's  scalp.  Leads 
from  these  electrodes  were  wired  to  our  instrumentation  package. 

Data  were  analyzed  on  our  Data  General  NOVA  computer  sys¬ 
tem.  The  system  has  a  dual  drive  floppy  disk,  a  custom  8-chan- 
nel  integrated  amplifier  and  filter  network  and  an  alpha-num¬ 
eric  oscilloscope  monitor.  We  have  since  upgraded  and  supple¬ 
mented  this  equipment  considerably,  and  now  our  laboratory  has 
what  we  consider  to  be  one  of  the  most  advanced  packages  of 
hard- and  software  evoked  potential  instrumentation  available 
anywhere.  The  setup  that  we  used  for  our  aviator  study  now 
seems  somewhat  primitive  by  comparison,  yet  the  results  we 
got  were  quite  interesting  and  quite  promising. 

Figure  5  shows  the  computer  output  for  a  single  subject. 
Actually,  our  upgraded  NOVA  system  now  produces  a  much  more 
sophisticated  and  informative  readout,  but  Figure  5  shows 
the  output  as  gathered  during  the  aviator  study. 

The  brain  waves  were  gathered  while  the  subject  was 
stimulated  by  a  series  of  100  light  flashes.  Because  the 
stimulus  is  a  visual  one,  these  brain  waves  are  referred  to 
as  visual  evoked  potentials.  For  each  of  the  eight  electrode 
sites,  we  have  the  averaged  wave  form  for  both  the  first  50 
flashes  and  the  second  50  flashes  (to  provide  a  measure  of 
habituation).  We  also  have  recorded,  to  the  side  of  each 
wave  form,  the  microvolt  root  mean  square  measure  of  ampli¬ 
tude  for  the  first  50  flashes  (top)  and  the  second  50 
flashes  (bottom).  On  the  left  hand  side  of  the  figure  you 
see  the  various  readings  for  the  left  hemisphere,  on  the 
right,  for  the  right  hemisphere.  The  top  two  recordings 
are  for  the  frontal  region  of  the  brain;  the  second  line 
across  is  for  the  central  region  of  the  brain,  left  and 
right  hemisphere  respectively.  On  the  third  line  across 
are  the  data  for  the  parietal  region  of  the  brain ,  and 
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Figure  3.  NPRDC  testing  van  In 


Figure  4.  Subject  wearing  Lycra  electrode  helmet. 
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Figure  5/ -Evoked  potential  redordings  from  eight 
.  electrode'  sites.' 


finally  at  the  bottom  are  the  data  for  the  occipital  elec¬ 
trodes,  that  is,  from  readings  taken  from  the  scalp  at  the 
back  of  the  head. 

We  asked  the  operations  officer  of  the  squadron,  who 
was  well  acquainted  with  the  proficiency  level  of  the  men 
to  provide  us  with  a  rating,  on  a  10  point  scale,  of  how 
proficient  each  subject  was  considered  to  be.  The  ratings 
ranged  from  7-10  for  the  Instructors  and  from  6-9  for  the 
students.  We  had  decided  that  our  Initial  analyses  would 
assume  that  each  Individual's  EP  characteristics  were  the  re¬ 
sult  of  his  aptitudes,  not  of  his  experiences,  so  we  added  1 
point  to  the  scores  of  each  of  the  students  to  correct  for  what 
we  assumed  to  be  a  decrement  due  to  lack  of  experience. 

Results 


Obviously,  we  have  collected  a  massive  amount  of  infor¬ 
mation  and  it  would  take  too  long  to  present  and  describe 
the  findings  in  any  detail.  For  present  purposes,  I  will 
restrict  the  analysis  to  two  major  questions  of  interest. 

(1)  Are  we  able  to  discriminate  the  pilot  and  RIO  groups 
on  the  basis  of  their  evoked  potentials?  (2)  Do  pilots 
and  RIOs  differ  in  terms  of  hemispheric  asymmetry,  and  if 
so,  are  the  differences  more  pronounced  for  those  pilots 
and  RIOs  rated  as  superior  performers  than  for  pilots  and 
RIOs  who  were  given  low  performance  ratings? 


With  regard  to  the  first  question,  on  the  group  differ¬ 
ences  between  pilots  and  RIOs,  Figure  6  presents  the  means 
and  standard  deviations  of  the  EP  amplitudes  measured  at  four 
sites  on  the  left  hemisphere.  As  you  can  see,  the  pilots 
were  more  variable  in  their  amplitudes,  and  also  had  greater 
mean  amplitudes,  then  did  the  RIOs,  at  each  of  the  four 
electrode  positions.  The  right  hemisphere  (Figure  7)  data 
showed  essentially  similar  findings.  In  no  case  was  the 
standard  deviation  or  the  mean  amplitude  for  the  RIO  group 
greater  than  for  the  pilot  group. 

Figure  8  is  a  scatter  plot  with  EP  amplitudes  from  the 
left  frontal  region  on  one  axis  and  EP  amplitudes  from  the 
left  central  region  of  the  brain  on  the  other  axis.  These 
variates  were  determined  by  discriminant  analysis .  The  dis¬ 
crimination  line,  which  was  drawn  in  visually  to  fit  these 
data,  suggests  that  one  might  be  able  to  predict  with  a 
fairly  high  degree  of  accuracy  whether  a  given  individual  is 
a  pilot  or  a  RIO  from  his  brain  wave  recordings.  Of  course. 
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Figure  7.  RIGHT  HEMISPHERE  MEANS  AND  STANDARD 

DEVIATIONS  AT  EACH  SITE 

'  nsc, 


X  PILOT  (N=28) 
A  RIO  (N=30) 


SCATTERPLOT  SHOWING  SEPARATION  OF  PILOTS  ANO  RIOS 


these  findings  will  have  to  be  cross-validated  on  another  sam¬ 
ple  before  we  could  have  much  reliance  In  them.  Pranklv  the 

“d  RIOS  can  be  separated  b^Se^^  eJok- 
ed  potentials  Is  rather  surprising  to  us.  It  did  not  seL  tn 
us,  from  what  we  were  able  to  learn  about  the  procedures  for 

assigning  pilots  and  RIOs,  that  there  would  be  so 
much  difference  In  the  evoked  potential^  of  th^wo  groups 
analysis  suggests  that  part,  but  not  all,  of  the 
oilots^and^RTn^  result  from  differing  cognitive  demands  upon 
that  questloJ)!'  reanalyzed  to  examine 

Next,  we  tested  the  hypothesis  that  the  cerebral  asvm- 

be  related^to^the^dlfferene^^^i'^^^^^c*  asymmetry  would 

,  ■  Qlff6PGnc6s  in  p6r*for*in3.ncG  rstincs  wlt^hin 

'J.-'Ieht  add  that  in  making  thesran^ysUr^ 
neeii  ^  the  subjects  who  were  left  handed  because  left-handed 
people  sometimes  have  reversed  dominance). 

be  relatlvPiv^L??®  Figure  9,  our  hypothesis  seemed  to 

be  relatively  well-confirmed.  All  of  the  high  Derformine 

right  hemisphere  amplitude  greater 
than  left  hemisphere  amplitude,  and  the  percentage  decreased 

was  ?rurfo?™?Ss!  decreased.  Exactly  the  opposite 

discussed  only  lateral  asymmetry~the 
right-left  distinction.  We  have  also  explored  front  to  back 
?f^urf?S’  some  interesting  results"  as  presenterin 

Figure  10  is  a  bit  complicated,  dealing  as  it  does 

*he  data  for  pilots?  we  s?? 

that  the  high  performing  men  showed  less  variation  In  thel?  EP 
asymmetry  than  the  low  performing  men,  and  that  this  difference 
n  variability  was  strikingly  more  pronounced  in  the  electrode 
the  back  of  the  head  (visual  area)  than  the  front. 

The  same  pattern  is  seen  for  RIOs,  and  also  for  a  group  of  28 

'warfare  (ASW)  trainees  on  whom  we  obtained 
similar  data  as  part  of  another  study. 

Discussion 

The  above  analyses  are,  of  course,  based  on  rather  small 

that^the  discriminant  analysis,  we  know 

positive  findings  we  report  are  affected  by  capitali- 

infr,  chance.  The  data  set  was  too  small  to  permit  division 

into  the  usual  training  and  testing  subpopulations.  However 
as  indicated  earlier,  this  was  intended  to  be  merely  an  explora¬ 
tory  study,  designed  to  try  out  our  gear,  to  see  hoi  effl^tivel^ 
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our  equipment  would  work  in  a  field  situation,  and  to  test 
out  some  hypotheses  that  we  hope  later  to  test  in  a  more 
formal  way  on  a  considerably  larger  sample. 

Additionally,  we  are  keenly  interested  in  finding  if  our 
results  can  be  confirmed  in  an  unrestricted  sample  of  trainees, 
before  those  with  the  lowest  aptitude  are  eliminated,  and  be¬ 
fore  Navy  training  and  experience  can  affect  the  results. 

We  are  only  beginning  to  Investigate  the  application  of 
psychobiologlcal  technology  to  the  problem  of  improving  per¬ 
sonnel  selection  and  training,  but  so  far,  we  have  been  en¬ 
couraged  by  what  we  have  found. 
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Abstract 


This  paper  presents  a  method  for  computing  the  proba¬ 
bility  of  an  aircrew  being  able  to  visually  locate  a  ground 
target  and  launch  a  weapon  against  it.  The  major  factors 
used  in  the  computations  are  target  acquisition  performance/ 
aircraft  maneuvering  requirements,  terrain  masking,  visi¬ 
bility,  and  weapon  operating  time.  Estimates  of  these 
factors  are  based  on  real-world  data  whenever  possible,  as 
Opposed  to  mathematical  modeling.  The  algorithm  used  to 
combine  these  factors  is  described  and  sample  results  are 
presented.  The  results  show  that  the  probability  of 
releasing  or  launching  a  weapon  on  a  target  is  quite  low 
in  many  situations. 
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I 


Introduction 


Most  air-to-ground  weapons  currently  in  use  require 
that  the  aircrew  make  a  visual  acquisition  of  the  target 
before  the  weapon  can  be  employed.  Such  weapons  include 
bombs,  guns,  rockets,  and  guided  missiles.  The  choice  of 
tactics  and  weapons,  and  the  estimation  of  the  effectiveness 
of  the  weapons  is  currently  based  upon  delivery  accuracy  and 
weapons  (warhead)  effectiveness  on  specific  targets.  The 
probability  of  finding  the  target  in  time  to  Convert  to  an 
attack  and  launch  the  weapon  almost  always  is  ignored. 

This  paper  presents  a  method  for  computing  the  proba¬ 
bility  of  an  aircrew  being  able  to  visually  locate  a  target, 
convert  to  an  attack  pass,  and  launch,  release,  or  fire  a 
weapon  against  the  target;  example  results  are  also  given. 

A.  Objective 

The  algorithm  described  here  was  developed  to  make 
it  possible  tO  estimate  the  probability  of  successfully 
making  a  first-pass  attack  on  a  ground  target  with  a  fixed- 
wing,  high-speed  aircraft.  The  probability  that  is  calcu¬ 
lated  describes  the  estimated  frequency  of  ns&,  or  utility 
of  a  given  aircraft/weapon  system  combination. 

Some  example  questions  that  might  be  answered  by 
this  probability  calculation  (or  measure  of  utility)  are: 

1.  What  percent  of  a  large  number  of  first-pass 
attacks  would  be  successful  against  a  column  of  tanks  moving 
in  European  terrain  during  the  day  in  June? 

2.  How  often  can  we  expect  to  successfully 
employ  a  gun,  a  missile,  or  a  bomb  against  three  tanks  in  a 
group  in  the  desert  in  December? 

B.  Limitations 

In  addition  to  the  limitation  of  the  algorithm  to 
the  utility  aspect  of  weapon  delivery,  there  are  other 
limitations  to  the  algorithm  in  its  present  form.  These 
are: 

1.  The  algorithm  is  limited  to  weapon  delivery 
by  high-speed,  fixed-wing  aircraft.  This  limitation  is 
present  because  the  data  used  in  the  algorithm  were  collected 
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in  field  tests  using  such  aircraft.  Extrapolation  to  other 
be  risky”^  helicopters  with  a  pop-up  ttiaheuvet)  would 

^l^orithm  is  limited  to  weapon  delivery 
involving  limited  maneuvers  by  the  delivering  aircraft. 

algorithm  are  derived  from 
toward  the  target  area;  pop-ups 
or  roll-ins  from  high  altitude  are  not  included  in  the 
calculations.  The  algorithm  best  describes  the  low-level, 
high-speed  delivery  tactic.  xev«x, 

^  subjective  estimate  has  been  made  oh 
the  limits  on  the  parameters  that  should  be  used.  These 

They^Sl^  serve  as  a  guideline  to  the  user,  however. 

Aircraft  altitude  -  500  to  2500  feet 
Aircraft  speed  -  350  to  550  knots 

II.  Method 


The  basic  approach  used  in  the  development  of  the 
algorithm  was  to  use  empirical  data  as  much  as  possible.  A 
means  was  devised  to  combine  this  empirical  data,  complement 

the'^desired°result^  calculations  when  required,  and  calculate 

empirical  data  was  preferred  since  it 
.  was  felt  that  such  data  is  more  representative  of  the  real 
theoretical  calculations.  Hence,  actual  ground 
survey  data  produced  from  optical  measurements  made  in  the 
field  was  preferred  to  map  study  results  for  the  computation 

Sight  (CLOS)  .  Field  test  results  giving 
visual  detection  ranges  of  ground  targets  by  pilots  were 
used  instead  of  a  sophisticated  mathematical  model  of  the 
geometry  and  the  visual  search  process. 

The  method  in  which  the  data  are  combined  in  the  alqo- 
rithm  IS  shown  in  Figure  1.  The  time  required  to  operate 
the  weapon  system  and  how  the  aircraft  will  be  flown  are 
used  to  calculate  the  Required  Range.  This  range  is  the 

which  the  pilot  must  visually  detect  the  target  in 
order  to  be  able  to  make  a  first-pass  attack,  if  the  pilot 

beyond  this  required  range,  he  will  be 
fu  ^  make  the  attack;  if  he  detects  the  target  closer 
than  the  required  range,  he  will  not  be  able  to  attack  on 
f p&ss • 
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Some  general  rules  have  been  derived  from  actual  field 
test  data  for  computing  the  probability  of  visually  detecting 
the  target.  The  computation  is  in  the  form  of  a  ciamulative 
probability  as  a  function  of  range  from  the  target.  The 
cumulative  probability  function  is  then  combined  with  the 
required  range  to  produce  the  probability  of  acquisition  hy 
the  required  range. 

The  last  procedure  in  the  algorithm  is  to  combine  the 
acquisition  probability  with  the  distribution  of  unmask 
ranges  and  visibility  ranges  actually  expected  to  occur  in 
the  region  of  interest.  Ceiling  data  can  also  be  used  to 
estimate  the  percent  of  the  time  that  particular  altitudes 
could  be  flown. 

This  last  procedure  produces  the  final  output  of  the 
algorithm:  the  expected  proportion  of  the  time  that  a  given 

target  can  be  successfully  attacked  under  a  set  of  specific 
conditions. 

A.  Aircraft  Flight  Parameters 

The  delivery  tactics  are,  in  part,  determined  by 
the  weapon  characteristics.  The  use  of  free-fall  bombs, 
guns,  or  unguided  missiles  requires  that  the  aircraft  be 
flown  directly  toward  the  target.  Other  weapons  with  some 
off-boresight  capability  have  also  been  used  principally  in 
the  straight-ahead  delivery  mode. 

Unless  exact  navigation,  or  target  cueing,  is  avail¬ 
able,  most  targets  should  be  expected  to  appear  somewhere 
off  the  dead-ahead  direction.  In  these  cases,  the  pilot  will 
be  required  to  turn  the  aircraft  toward  the  target  before 
preparing  for  weapon  release.  The  geometry  describing  the 
entire  attack  process  is  shown  in  Figure  2. 


The  range  required  to  make  the  attack  decision  and 
roll  the  aircraft  is  designated  A  in  Figure  2(c).  After  the 
turn  is  complete,  the  aircraft  must  be  rolled  level,  the 
weapon  must  be  readied  for  launch,  and  launched  some  minimum 
range  from  the  target.  These  events  are  included  in  the 
straight  segment,  B,‘  in  Figure  2(c)  . 


From  the  geometry  of  Figure  2(c)  one  can  show  that 

2 

=  (A  cos  a  +  r  sin  a)  ±  [(A  cos  a  +  r  sin  ct) 


-  (a2  -  b2]V2 


(1) 
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The  turning  radius  of  an  aircraft  is  given  by 


where  g  is  the  gravitational  constant.  Other  substitutions 
that  can  be  made  in  Equation  1  are  related  to  the  terms 
discussed  above. 

The  factors  discussed  above  can  be  included  in 
Equation  1  by  substituting 


The  weapon  operating  time,  T^p  given  in  Equation 

4,  is  determined  by  the  weapon  system  characteristics,  the 
aircrew's  capabilities,  and  the  environmental  operating 
conditions.  Operating  times  can  simply  be  assumed,  derived 
from  manned  simulation  tests,  or  from  flight  tests.  The 
times  have  been  found  to  vary  from  2  sec.  to  as  much  as 
12  sec. 

The  examples  of  operating  times  found  in  the 
literature  illustrate  the  wide  range  of  times  that  might  be 
required  with  different  aircraft  systems.  Another  factor 
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that  might  affect  these  times  is  the  size  of  the  crew;  it  is 
thought  that  a  single  pilot  would  reqUire  more  time  to 
operate  a  complicated  weapon  system  than  an  aircrew  of  two. 

The  pilot  must  operate  the  system  as  well  as  fly  the  aircraft. 

1 .  Anqle-Of f 

The  distribution  of  angle-off  is  a  function 
Of  the  accuracy  of  the  intelligence  information,  the  air¬ 
craft's  navigation  system,  the  target's  mobility,  avail¬ 
ability  of  external  target  designation  (e.g. ,  forward  air 
controller),  and  many  other  variables.  No  data  sources  have 
been  located  to  date  that  could  be  used  to  derive  angle-dff 
distributions,  so  assumptions  must  be  made  if  a  distribution 
is  used.  Suffice  it  to  say  that  use  of  the  algorithm  does 
not  require  the  assuinption  that  the  target  will  always 
appear  straight  ahead  of  the  aircraft. 

2.  Flexibility  in  Required  Range  Computation 

The  parameters  in  the  computation  have  been 
named:  Tj^^  is  called  roll-in  time,  Tj^  is  decision  time, 

etc.  Other  sequences  of  operation  may  require  other  events 
to  occur,  and  the  formulation  given  in  Equation  1  can  be 
used  by  setting  some  values  to  zero  and/or  changing  the 
names  of  events.  As  long  as  the  situation  of  interest  has  a 
straight-line  segment,  a  curve  representing  the  turning 
aircraft  and  another  straight-line  segment.  Equation  1  can 
be  used. 

B.  Visual  Target  Acquisition 

The  next  step  in  the  algorithm  is  the  computation 
of  the  probability  that  the  pilot  will  see  the  target  as  he 
flies  toward  the  target  area.  The  result  is  a  cumulative 
probability  as  a  function  of  range,  for  a  given  target/ 
background  combination. 

1.  Background 

Two  separate  study  efforts  led  to  the  develop¬ 
ment  of  the  technique  for  computing  acquisition  probability: 
evaluation  of  mathematical  models  and  sximmary  of  field  test 
data.  The  model  evaluation  effort  illustrated  that  there 
are  often  large  differences  among  the  many  models  that  have 
been  developed.  It  also  showed  that  the  models  have  not 
often  been  validated  by  field  tests,  so  that  one  does  not 
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know  which  of  the  models  is  the  |est  predictor  of  target 
acquisition  performance.  \ 


The  summary  of  field 'tesfr  data  provided  a  descrip- 
tion  of  over  45  field  tests  of  target  acquisition  and  sample 
results  of  the  tests.  This  tabulation  of  results  illustrated 
that  some  actual  test  data  were  available  for  use  in  makina 
performance  predictions. 


The  result  of  this  effort  is  a  comparatively  simple 
model  for  computing  target  acquisition  performance.  The 

model  IS  really  a  data  fit,  and  is  based  upon  actual  field 
test  data. 


2*  Target  Acquisition  Definition 

.  The  definitions  of  target  detection,  identi¬ 

fication,  recognition,  classification,  and  acquisition  have 
been  discussed  and  given  in  many,  many  reports  on- the  subject. 
This  simplified  model  is  based  on  data  from  different  field 
tests,  where  performance  measures  were  not  accurately  defined. 
The  target  acquisition  response  seemed  to  be  "I  see  the 

have  the  target  in  sight."  it  seemed  to  be 
the  point  at  which  the  pilot  saw  enough,  or  had  enough 
information,  to  be  willing  to  begin  an  attack  pass  on  the 
Object.  This  very  general  definition  is  the  one  used  in 
the  simplified  model. 

3*  Target  Acquisition  Probability 

The  computation  procedure  uses  subjective 
estimates  of  the  visual  appear arige  of  the  target  as  well  as 
physical  measurements  (or  estimatos)  of  the  target  size, 
masking,  and  visibility. 

The  conspicuousness  characteristic  of  the 
target  is  expressed  in  two  ways:  " contras tiness"  and  "asso¬ 
ciated  pattern."  The  contrastiness  of  the  target  is  the 
visual  contrast  between  the  most  significant,  distinctive, 
target-related  feature  and  its  background.  The  contrasting 
element  may  be  the  target  object  i,tself,  or  a  distinctive 
associated  feature. 


.  ^  The  associated  pattern  is  the  target-related 

pattern  in  the  target  area.  The  pattern  may  be  made  up  of 
target  elements  (e.g. ,  a  straight  row  of  trucks)  or  of  other 
elements  (roads,  a  river)  that  can  be  associated  with  the 
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The  maximiun  probability  of  target  acquisition 
is  taken  from  Table  1  as  a  function  of  the  estimates  of 
contrastiness  and  pattern. 


TABLE  1.  Maximum  Sighting  Probability, 


Pattern 

Contrastiness  | 

High 

Medium 

Low 

Large 

1.00 

0.75 

0 . 50 

Medium 

0.75 

0.56 

0.37 

Small 

0.50 

0.37 

0.25 

4.  Target  Acquisition  Range 

The  probability  of  acquiring  the  target  as  a 
function  of  range  is  ass\amed  to  be  related  to  the  point  at 
which  the  target  becomes  optically  available.  The  point  at 
which  the  target  is  unmasked  to  the  observer  (where  a  clear 
line-of-sight  exists)  and  the  meteorological  range  (visi¬ 
bility)  are  the  major  variables. 

The  rules  of  thumb  that  were  derived  from 
flight  test  data  are  as  follows: 

a.  The  median  range  of  acquisition  will 
occur  at  one- half  the  unmask  range,  or  one-half  the  meteoro¬ 
logical  range,  whichever  is  smaller. 

b.  The  probability  of  acquisition  will  be 
0.2  and  0.8  of  the  value  taken  from  Table  2  at  0.625  and 
0.375,  respectively,  of  the  unmask  range  or  meteorological 
range,  whichever  is  smaller. 

These  rules  of  thumb  make  it  possible  to  construct  a  curve 
similar  to  that  shown  in  Figure  3. 

The  algorithm  uses  the  equation 
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to  fit  the  curve,  where 

probability  of  acquisition 
maximtim  probability  taken  from  Table  1 
required  range  (Equation  1) 
meteorological  range  of  unmask  range. 


Equation  5  and  the  curve  shown  in  Figure  3 
are  functions  of  the  unmask  range  or  the  meteorological 
range  (visibility).  At  this  point  in  the  computation  pro¬ 
cess,  the  probability  curves  are  generated  for  specific 
distributions  or  values  of  target  type,  weapon  type  (oper¬ 
ating  time) ,  aircraft  velocity,  and  initial  target  angle- 
off.  It  remains  to  modify  the  calculations  by  the  unmask 
and  visibility  data  actually  expected  in  the  area  of  interest. 

C.  Masking,  Ceiling,  and  visibility  Data 

The  environmental  data  included  in  the  algorithm 
tie  the  probability  of  launch  calculation  to  a  specific  time 
and  place  by  using  representative  masking  and  visibility 
data.  The  data  are  used  to  weight  the  probability  calcula¬ 
tion  made  by  Equation  3  by  the  expected  frequency  of  occur¬ 
rence  of  masking,  visibility,  artd  ceiling  values. 

1.  Masking 

The  masking  data  used  in  the  algorithm  were 
produced  by  an  actual  ground  survey,  and  include  both  terrain 
and  vegetation  effects.  These  data  are  stored  in  the  algo¬ 
rithm  and  used  to  compute  probability  of  Unmask  for  whatever 
range  and  aircraft  altitude  the  user  chooses.  The  computer 
file  contains  an  element  for  each  terrain  type;  designation 
of  the  code  name  causes  the  appropriate  masking  data  to  be 
used  in  the  computation.  The  user  may  also  use  other  masking 
data,  provided  such  data  are  in  the  form  of  mask  angles  and 
ranges  to  masking  objects. 

2.  .Visibility  and  Ceiling 

Weather  data  from  the  USAF  Environmental 
Technical  Applications  Center  (ETAC)  have  been  found  to  be 
the  most  comprehensive  source  for  algorithm  use.  The  data 
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are  usually  in  the  form  of  cumulative  probability  curves 
that  show  the  probability  that  visibility  is  equal  to  or 
greater  than  any  given  value,  or  that  ceiling  is  at  least  as 
high  as  a  given  altitude. 

3.  Use  of  the  Data 

The  algorithm  converts  these  cumulative 
curves  into  discrete  distributions  of  probability.  The 
discrete  probabilities  are  then  each  multiplied  by  the 
acquisition  probability  computed  from  Equation  3  with  set 
equal  to  the  discrete  range. 

In  concept.  Equation  3  gives  the  probability 
that  the  aircrew  can  convert  to  a  launch  if  the  unmask  range 
or  visibility  is  R^.  This  probability  is  then  multiplied  by 

the  probability  of  R^  occurring  to  estimate  how  often  a 

launch  can  occur.  By  summing  all  these  products  together, 
the  entire  time  period  is  covered  (the  discrete  probabilities 
must  add  to  1.0). 

The  ceiling  data  are  also  entered  as  a  cumula¬ 
tive  probability  of  the  ceiling  being  at  least  as  high  as  a 
given  altitude.  The  user  may  operate  the  program  without 
ceiling  being  included  (i.e.,  the  assumption  of  a  clear 
sky),  or  with  a  ceiling  calculation.  The  effect  of  the 
latter  is  to  multiply  the  probability  of  a  launch  by  the 
probability  of  being  able  to  fly  at  the  chosen  altitude. 

III.  Sample  Results  and  Sensitivity 

This  section  of  the  report  presents  some  sample  results 
from  the  algorithm.  Not  all  of  the  variables  were  changed 
for  the  sample  runs;  those  held  constant  are  shown  in  Table 
2.  These  weather  conditions  were  used  in  the  computations; 
for  convenience  in  later  referencing  they  are  referred  to 
simply  as  locations  A,  B,  and  C.  The  weather  at  locations  A 
and  B  is  similar,  and  would  be  judged  good  flying  weather, 
both  winter  and  summer.  The  weather  at  location  C  is  worse, 
with  much  lower  ceilings  and  poorer  visibility  in  the  winter. 


The  terrains  chosen  for  the  sample  runs  illustrate  the 
variety  to  be  expected,  from  flat,  open  terrain  to  sharply 
rolling  terrain. 
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TABLE  2.  Variables  Helf’ Constant  in  Sample 
Results  Presen^o^  pelow. 


Decision  time,  sec  .  .  >,* 
Roll-in  time,  sec  .  .  •  ? 
Aircraft  velocity,  knots 
Minimum  release  ran^e,  ft 
Niamber  of  g* s  in  turn  .  . 
Roll-out  time,  sec  .  . 


•  •  •  .  •  1 
•  •  •  •  0*5 

.  ...  450 

,  .  .  .  3,000 

.  .  .  .  3 

.  .  .  .  1.0 


A.  Target  Effects 

The  algorithm  has  a  large  built-in  target  effect 
since  the  user  must  select  the  estimated  acquisition  proba¬ 
bility,  from  Table  1;  the  values  range  from  0.25  to 

1.00.  This  effect  is  illustrated  in  Figure  4,  where 

values  of  1.00,  0.75,  0.37,  and  0.25  were  selected  for 
running.  The  resulting  probabilities  of  launch  range  from 
0.75  down  to  0.20;  there  is  a  direct  variation  in  when 

there  is  a  variation  in  variation  is  a  function 

of  the  algorithm  user's  estimate  of  how  hard  it  is  to  find 
the  target. 

B.  Terrain  Effects 

Figure  5  illustrates  the  effect  of  the  type  of 
terrain  on  the  probability  of  launch.  A  fairly  easy  target 
is  assumed  (P^  =  0.75)  and  the  p^robability  of  launch  is 

about  0.50  in  flat,  open  terrain.  The  launch  probability  is 
only  0.05  in  shaprly  rolling  terrain  when  the  aircraft  is 
flying  at  low  altitude,  and  increases  to  only  0.25  at  an 
altitude  of  4,000  ft. 

This  large  terrain  effect  is  produced  by  target 
masking  by  the  terrain  and  vegetation.  Although  the  target 
will  be  seen  on  75%  of  the  passes,  it  is  seen  too  late  to 
get  off  a  launch  on  most  of  the  |;»asses.  The  major  factor 
that  interacts  with  the  terrain  (effect  is  the  aircraft 
altitude  discussed  in  the  next  section. 
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C .  Altitude  Effects 

Figure  6  shows  the  probability  of  being  able  to 
fly  and  launch  a  weapon  from  different  altitudes  under  two 
weather  conditions  (December  and  June)  in  two  different 
types  of  terrain.  A  target  acquisition  probability  of  1.0 
was  assumed. 

In  flat/  open  terrain  with  good  weather,  the 
probability  increases  considerably  whdn  the  aircraft  goes 
from  500  to  1,000  feet.  There  is  hot  much  improvement  above 
1,000  feet;  and,  in  fact,  there  is  a  slight  decrement  because 
some  of  the  time  the  ceiling  will  be  below  the  flight  alti¬ 
tude  . 

In  the  same  terrain  with  poor  weather,  the  proba¬ 
bility  of  launch  decreases  with  altitude.  Visibility  causes 
the  degradation  and  the  probability  of  a  clear  sky  at  alti¬ 
tude  gets  lower  the  higher  one  gets. 

The  launch  probability  continually  increases  with 
altitude  in  sharply  rolling  terrain  with  good  weather; 
masking  is  the  cause  of  the  degradation  in  this  case,  and 
the  higher  the  aircraft  flies,  the  better  the  chances  of  a 
clear  line-of-sight. 

In  summary,  increasing  the  planned  attack  altitude 
can  either  increase  or  decrease  the  percent  of  the  time  an 
attack  can  be  made.  Increasing  the  altitude  overcomes  mask¬ 
ing  problems,  but  may  put  the  aircraft  in  the  clouds.  The 
weather  and  type  of  terrain  must  be  known  to  determine  the 
major  effect. 
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PROBABILITY  OF  LAUNCH 


AIRCRAFT  ALTITUDE,  FT 


FIGURE  4.  Probability  of  Launch 
as  a  Function  of  Target  Acquisition 
Difficulty  for  Weather  at  Location  B 
in  June  (Figures  10  and  1 1)  With 
Top  =  7  sec  and  Angle-Off  =15  deg. 
Regions  of  predominant  terrain  and 
ceiling  effects  are  shown  on  the 
curves. 
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